[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:39:32 dist-test-slave-zpbg krb5kdc[6092](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:39:32 dist-test-slave-zpbg krb5kdc[6092](info): set up 2 sockets
Jun 19 02:39:32 dist-test-slave-zpbg krb5kdc[6092](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260619 02:39:33.912282 6085 mini_kdc.cc:121] Time spent starting KDC: real 1.041s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Jun 19 02:39:33 dist-test-slave-zpbg krb5kdc[6092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836773, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:39:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:39:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.241.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
I20260619 02:39:34.029378 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:36941
--webserver_interface=127.5.241.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:46309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:36941
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:39:34.109422 6105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:39:34.109635 6105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:39:34.109658 6105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:39:34.110765 6105 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:39:34.110828 6105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:39:34.110843 6105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:39:34.110855 6105 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:39:34.110868 6105 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:39:34.112282 6105 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:46309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:36941
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:36941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.241.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6105
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:39:34.112478 6105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:39:34.112649 6105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:39:34.117998 6110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:34.118011 6113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:39:34.118024 6105 server_base.cc:1034] running on GCE node
W20260619 02:39:34.118005 6111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:39:34.118630 6105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:39:34.118968 6105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:39:34.120123 6105 hybrid_clock.cc:648] HybridClock initialized: now 1781836774120105 us; error 40 us; skew 500 ppm
Jun 19 02:39:34 dist-test-slave-zpbg krb5kdc[6092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836774, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:39:34.122849 6105 init.cc:377] Logged in from keytab as kudu/127.5.241.126@KRBTEST.COM (short username kudu)
I20260619 02:39:34.123314 6105 webserver.cc:460] Webserver started at http://127.5.241.126:39201/ using document root <none> and password file <none>
I20260619 02:39:34.123517 6105 fs_manager.cc:359] Metadata directory not provided
I20260619 02:39:34.123558 6105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:39:34.123656 6105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:39:34.124720 6105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/master-0/data/instance:
uuid: "99551a72bca9498fb834114331c6c330"
format_stamp: "Formatted at 2026-06-19 02:39:34 on dist-test-slave-zpbg"
server_key: "23555f797cf276060cfe131f69e2fb05"
server_key_iv: "86d73981489bacb1f884b856f5d049da"
server_key_version: "encryptionkey@0"
I20260619 02:39:34.125039 6105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/master-0/wal/instance:
uuid: "99551a72bca9498fb834114331c6c330"
format_stamp: "Formatted at 2026-06-19 02:39:34 on dist-test-slave-zpbg"
server_key: "23555f797cf276060cfe131f69e2fb05"
server_key_iv: "86d73981489bacb1f884b856f5d049da"
server_key_version: "encryptionkey@0"
I20260619 02:39:34.126348 6105 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:39:34.127085 6120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:34.127298 6105 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:39:34.127398 6105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "99551a72bca9498fb834114331c6c330"
format_stamp: "Formatted at 2026-06-19 02:39:34 on dist-test-slave-zpbg"
server_key: "23555f797cf276060cfe131f69e2fb05"
server_key_iv: "86d73981489bacb1f884b856f5d049da"
server_key_version: "encryptionkey@0"
I20260619 02:39:34.127482 6105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:39:34.148188 6105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:39:34.150241 6105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:39:34.150415 6105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:39:34.154508 6105 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.126:36941
I20260619 02:39:34.154548 6172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.126:36941 every 8 connection(s)
I20260619 02:39:34.154965 6105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:39:34.155856 6173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:39:34.158207 6173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99551a72bca9498fb834114331c6c330: Bootstrap starting.
I20260619 02:39:34.158865 6173 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 99551a72bca9498fb834114331c6c330: Neither blocks nor log segments found. Creating new log.
I20260619 02:39:34.159154 6173 log.cc:824] T 00000000000000000000000000000000 P 99551a72bca9498fb834114331c6c330: Log is configured to *not* fsync() on all Append() calls
I20260619 02:39:34.159852 6173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99551a72bca9498fb834114331c6c330: No bootstrap required, opened a new log
I20260619 02:39:34.161466 6173 raft_consensus.cc:348] T 00000000000000000000000000000000 P 99551a72bca9498fb834114331c6c330 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99551a72bca9498fb834114331c6c330" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 36941 } }
I20260619 02:39:34.161607 6173 raft_consensus.cc:374] T 00000000000000000000000000000000 P 99551a72bca9498fb834114331c6c330 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:39:34.161630 6173 raft_consensus.cc:729] T 00000000000000000000000000000000 P 99551a72bca9498fb834114331c6c330 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99551a72bca9498fb834114331c6c330, State: Initialized, Role: FOLLOWER
I20260619 02:39:34.161783 6173 consensus_queue.cc:260] T 00000000000000000000000000000000 P 99551a72bca9498fb834114331c6c330 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99551a72bca9498fb834114331c6c330" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 36941 } }
I20260619 02:39:34.161842 6173 raft_consensus.cc:388] T 00000000000000000000000000000000 P 99551a72bca9498fb834114331c6c330 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:39:34.161888 6173 raft_consensus.cc:482] T 00000000000000000000000000000000 P 99551a72bca9498fb834114331c6c330 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:39:34.161947 6173 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 99551a72bca9498fb834114331c6c330 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:39:34.162442 6173 raft_consensus.cc:504] T 00000000000000000000000000000000 P 99551a72bca9498fb834114331c6c330 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99551a72bca9498fb834114331c6c330" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 36941 } }
I20260619 02:39:34.162524 6173 leader_election.cc:302] T 00000000000000000000000000000000 P 99551a72bca9498fb834114331c6c330 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 99551a72bca9498fb834114331c6c330; no voters:
I20260619 02:39:34.162648 6173 leader_election.cc:288] T 00000000000000000000000000000000 P 99551a72bca9498fb834114331c6c330 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:39:34.162711 6176 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 99551a72bca9498fb834114331c6c330 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:39:34.162892 6176 raft_consensus.cc:686] T 00000000000000000000000000000000 P 99551a72bca9498fb834114331c6c330 [term 1 LEADER]: Becoming Leader. State: Replica: 99551a72bca9498fb834114331c6c330, State: Running, Role: LEADER
I20260619 02:39:34.162937 6173 sys_catalog.cc:564] T 00000000000000000000000000000000 P 99551a72bca9498fb834114331c6c330 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:39:34.162981 6176 consensus_queue.cc:237] T 00000000000000000000000000000000 P 99551a72bca9498fb834114331c6c330 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99551a72bca9498fb834114331c6c330" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 36941 } }
I20260619 02:39:34.163368 6178 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99551a72bca9498fb834114331c6c330 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 99551a72bca9498fb834114331c6c330. Latest consensus state: current_term: 1 leader_uuid: "99551a72bca9498fb834114331c6c330" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99551a72bca9498fb834114331c6c330" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 36941 } } }
I20260619 02:39:34.163453 6178 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99551a72bca9498fb834114331c6c330 [sys.catalog]: This master's current role is: LEADER
I20260619 02:39:34.163501 6177 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99551a72bca9498fb834114331c6c330 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "99551a72bca9498fb834114331c6c330" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99551a72bca9498fb834114331c6c330" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 36941 } } }
I20260619 02:39:34.163573 6177 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99551a72bca9498fb834114331c6c330 [sys.catalog]: This master's current role is: LEADER
I20260619 02:39:34.163667 6184 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:39:34.164364 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 6105
I20260619 02:39:34.164471 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/master-0/wal/instance
I20260619 02:39:34.165087 6085 external_mini_cluster.cc:1428] Setting key 097f755356d85c2c26d4393543c8d12f
I20260619 02:39:34.165993 6184 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:39:34.167680 6184 catalog_manager.cc:1347] Generated new cluster ID: 7ecca3c2bc254970b6eef0c44869fcba
I20260619 02:39:34.167733 6184 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 02:39:34 dist-test-slave-zpbg krb5kdc[6092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836773, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:39:34.183279 6184 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:39:34.183449 6194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:34.165887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45436 (local address 127.5.241.126:36941)
0619 02:39:34.167181 (+ 1294us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:34.167189 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:34.167360 (+ 171us) server_negotiation.cc:407] Connection header received
0619 02:39:34.168771 (+ 1411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:34.168778 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:34.168837 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:34.168972 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:34.171088 (+ 2116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:34.172488 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:34.174095 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:34.174704 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:34.174824 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:34.179185 (+ 4361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:34.179190 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:34.179190 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:34.179203 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:34.181305 (+ 2102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:34.181792 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:34.181796 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:34.181796 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:34.181909 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:34.182208 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:34.182209 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:34.182210 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:34.182530 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:34.182678 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:34.183002 (+ 324us) server_negotiation.cc:299] Negotiation successful
0619 02:39:34.183137 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1249,"thread_start_us":60,"threads_started":1}
I20260619 02:39:34.184340 6184 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:39:34.189569 6184 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 99551a72bca9498fb834114331c6c330: Generated new TSK 0
I20260619 02:39:34.189774 6184 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.241.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab.
Entry for principal kudu/127.5.241.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab.
I20260619 02:39:34.222090 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.65:0
--local_ip_for_outbound_sockets=127.5.241.65
--webserver_interface=127.5.241.65
--webserver_port=0
--tserver_master_addrs=127.5.241.126:36941
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab
--principal=kudu/127.5.241.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:46309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:39:34.328086 6199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:39:34.328321 6199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:39:34.328364 6199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:39:34.329526 6199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:39:34.329586 6199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:39:34.329632 6199 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.65
I20260619 02:39:34.331396 6199 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:46309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab
--principal=kudu/127.5.241.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.241.65
--webserver_port=0
--tserver_master_addrs=127.5.241.126:36941
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6199
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.65
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:39:34.331661 6199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:39:34.331912 6199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:39:34.337008 6199 server_base.cc:1034] running on GCE node
W20260619 02:39:34.336943 6207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:34.336979 6204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:34.336979 6205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:39:34.337440 6199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:39:34.337653 6199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:39:34.338814 6199 hybrid_clock.cc:648] HybridClock initialized: now 1781836774338800 us; error 28 us; skew 500 ppm
Jun 19 02:39:34 dist-test-slave-zpbg krb5kdc[6092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836774, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:39:34.341136 6199 init.cc:377] Logged in from keytab as kudu/127.5.241.65@KRBTEST.COM (short username kudu)
I20260619 02:39:34.341547 6199 webserver.cc:460] Webserver started at http://127.5.241.65:38503/ using document root <none> and password file <none>
I20260619 02:39:34.341786 6199 fs_manager.cc:359] Metadata directory not provided
I20260619 02:39:34.341851 6199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:39:34.341992 6199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:39:34.343160 6199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/ts-0/data/instance:
uuid: "8e05e70ba673460187197917d41a14d7"
format_stamp: "Formatted at 2026-06-19 02:39:34 on dist-test-slave-zpbg"
server_key: "e5ade631363882439d5350cdd758b54e"
server_key_iv: "2fc76147f1e6487c91fad0e54140f422"
server_key_version: "encryptionkey@0"
I20260619 02:39:34.343523 6199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/ts-0/wal/instance:
uuid: "8e05e70ba673460187197917d41a14d7"
format_stamp: "Formatted at 2026-06-19 02:39:34 on dist-test-slave-zpbg"
server_key: "e5ade631363882439d5350cdd758b54e"
server_key_iv: "2fc76147f1e6487c91fad0e54140f422"
server_key_version: "encryptionkey@0"
I20260619 02:39:34.344800 6199 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:39:34.345481 6214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:34.345647 6199 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:39:34.345747 6199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/ts-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/ts-0/wal
uuid: "8e05e70ba673460187197917d41a14d7"
format_stamp: "Formatted at 2026-06-19 02:39:34 on dist-test-slave-zpbg"
server_key: "e5ade631363882439d5350cdd758b54e"
server_key_iv: "2fc76147f1e6487c91fad0e54140f422"
server_key_version: "encryptionkey@0"
I20260619 02:39:34.345816 6199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:39:34.381822 6199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:39:34.384263 6199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:39:34.384438 6199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:39:34.384694 6199 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:39:34.385011 6199 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:39:34.385063 6199 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:34.385103 6199 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:39:34.385145 6199 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:34.391363 6199 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.65:42955
I20260619 02:39:34.391453 6327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.65:42955 every 8 connection(s)
I20260619 02:39:34.391788 6199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
Jun 19 02:39:34 dist-test-slave-zpbg krb5kdc[6092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836774, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.65@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:39:34.398208 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 6199
I20260619 02:39:34.398293 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/ts-0/wal/instance
I20260619 02:39:34.398451 6085 external_mini_cluster.cc:1428] Setting key cf87cc1b1c12a869b7797ae7fd729f64
I20260619 02:39:34.401513 6194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:34.392567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:56307 (local address 127.5.241.126:36941)
0619 02:39:34.392644 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:34.392646 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:34.392827 (+ 181us) server_negotiation.cc:407] Connection header received
0619 02:39:34.393385 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:34.393387 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:34.393404 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:34.393451 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:34.394467 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:34.395045 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:34.396068 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:34.396277 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:34.396360 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:34.398625 (+ 2265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:34.398627 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:34.398627 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:34.398643 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:34.400135 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:34.400562 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:34.400564 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:34.400564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:34.400631 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:34.400876 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:34.400877 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:34.400877 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:34.400993 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:34.401067 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:34.401332 (+ 265us) server_negotiation.cc:299] Negotiation successful
0619 02:39:34.401425 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 02:39:34.401717 6330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:34.392530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:36941 (local address 127.5.241.65:56307)
0619 02:39:34.392704 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:34.392748 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:34.393266 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:34.393534 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:34.393539 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:34.394143 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:39:34.394340 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:34.394346 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:34.395144 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:34.395145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:34.395964 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:34.395966 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:34.396372 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:34.396373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:34.396503 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:34.397031 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:39:34.397039 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:39:34.398522 (+ 1483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:39:34.400233 (+ 1711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:34.400240 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:34.400242 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:34.400474 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:34.400694 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:34.400694 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:34.400695 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:34.400810 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:34.401056 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:39:34.401062 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:39:34.401203 (+ 141us) client_negotiation.cc:769] Sending connection context
0619 02:39:34.401311 (+ 108us) client_negotiation.cc:240] Negotiation successful
0619 02:39:34.401473 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":82,"threads_started":1}
I20260619 02:39:34.402175 6328 heartbeater.cc:347] Connected to a master server at 127.5.241.126:36941
I20260619 02:39:34.402280 6328 heartbeater.cc:464] Registering TS with master...
I20260619 02:39:34.402467 6328 heartbeater.cc:511] Master 127.5.241.126:36941 requested a full tablet report, sending...
I20260619 02:39:34.403050 6137 ts_manager.cc:194] Registered new tserver with Master: 8e05e70ba673460187197917d41a14d7 (127.5.241.65:42955)
I20260619 02:39:34.404021 6137 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.65@KRBTEST.COM'} at 127.5.241.65:56307
WARNING: no policy specified for kudu/127.5.241.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab.
Entry for principal kudu/127.5.241.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab.
I20260619 02:39:34.426443 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.66:0
--local_ip_for_outbound_sockets=127.5.241.66
--webserver_interface=127.5.241.66
--webserver_port=0
--tserver_master_addrs=127.5.241.126:36941
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab
--principal=kudu/127.5.241.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:46309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:39:34.508941 6333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:39:34.509255 6333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:39:34.509292 6333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:39:34.510462 6333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:39:34.510528 6333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:39:34.510571 6333 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.66
I20260619 02:39:34.512019 6333 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:46309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab
--principal=kudu/127.5.241.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.241.66
--webserver_port=0
--tserver_master_addrs=127.5.241.126:36941
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6333
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.66
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:39:34.512281 6333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:39:34.512518 6333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:39:34.517576 6339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:34.517576 6341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:34.517604 6338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:39:34.517861 6333 server_base.cc:1034] running on GCE node
I20260619 02:39:34.518082 6333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:39:34.518321 6333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:39:34.519480 6333 hybrid_clock.cc:648] HybridClock initialized: now 1781836774519467 us; error 40 us; skew 500 ppm
Jun 19 02:39:34 dist-test-slave-zpbg krb5kdc[6092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836774, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:39:34.522064 6333 init.cc:377] Logged in from keytab as kudu/127.5.241.66@KRBTEST.COM (short username kudu)
I20260619 02:39:34.522485 6333 webserver.cc:460] Webserver started at http://127.5.241.66:42421/ using document root <none> and password file <none>
I20260619 02:39:34.522713 6333 fs_manager.cc:359] Metadata directory not provided
I20260619 02:39:34.522799 6333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:39:34.522946 6333 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:39:34.523979 6333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/ts-1/data/instance:
uuid: "ec89db343470413f888ef557a7f44f34"
format_stamp: "Formatted at 2026-06-19 02:39:34 on dist-test-slave-zpbg"
server_key: "1f95ce93f0b356200e56a7f75f4a87d6"
server_key_iv: "53dcbe896bad26c721e6be324c42bb6f"
server_key_version: "encryptionkey@0"
I20260619 02:39:34.524343 6333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/ts-1/wal/instance:
uuid: "ec89db343470413f888ef557a7f44f34"
format_stamp: "Formatted at 2026-06-19 02:39:34 on dist-test-slave-zpbg"
server_key: "1f95ce93f0b356200e56a7f75f4a87d6"
server_key_iv: "53dcbe896bad26c721e6be324c42bb6f"
server_key_version: "encryptionkey@0"
I20260619 02:39:34.525719 6333 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:39:34.526398 6348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:34.526562 6333 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:39:34.526656 6333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/ts-1/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/ts-1/wal
uuid: "ec89db343470413f888ef557a7f44f34"
format_stamp: "Formatted at 2026-06-19 02:39:34 on dist-test-slave-zpbg"
server_key: "1f95ce93f0b356200e56a7f75f4a87d6"
server_key_iv: "53dcbe896bad26c721e6be324c42bb6f"
server_key_version: "encryptionkey@0"
I20260619 02:39:34.526733 6333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:39:34.541734 6333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:39:34.543903 6333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:39:34.544061 6333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:39:34.544302 6333 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:39:34.544605 6333 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:39:34.544656 6333 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:34.544695 6333 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:39:34.544726 6333 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:34.551424 6333 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.66:43815
I20260619 02:39:34.551466 6461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.66:43815 every 8 connection(s)
I20260619 02:39:34.551789 6333 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
Jun 19 02:39:34 dist-test-slave-zpbg krb5kdc[6092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836774, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.66@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:39:34.561241 6194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:34.552538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:41503 (local address 127.5.241.126:36941)
0619 02:39:34.552602 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:34.552603 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:34.552770 (+ 167us) server_negotiation.cc:407] Connection header received
0619 02:39:34.553254 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:34.553255 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:34.553266 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:34.553336 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:34.554248 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:34.554742 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:34.555809 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:34.556029 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:34.556089 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:34.558345 (+ 2256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:34.558346 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:34.558346 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:34.558353 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:34.559844 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:34.560234 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:34.560235 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:34.560235 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:34.560302 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:34.560563 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:34.560564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:34.560564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:34.560672 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:34.560738 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:34.561017 (+ 279us) server_negotiation.cc:299] Negotiation successful
0619 02:39:34.561126 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 02:39:34.561254 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 6333
I20260619 02:39:34.561316 6464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:34.552484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:36941 (local address 127.5.241.66:41503)
0619 02:39:34.552635 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:34.552676 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:34.553143 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:34.553392 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:34.553396 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:34.553957 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:39:34.554156 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:34.554161 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:34.554870 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:34.554871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:34.555719 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:34.555722 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:34.556108 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:34.556109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:34.556214 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:34.556772 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:39:34.556781 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:39:34.558261 (+ 1480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:39:34.559923 (+ 1662us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:34.559929 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:34.559931 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:34.560155 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:34.560364 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:34.560365 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:34.560365 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:34.560503 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:34.560757 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:39:34.560763 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:39:34.560895 (+ 132us) client_negotiation.cc:769] Sending connection context
0619 02:39:34.560997 (+ 102us) client_negotiation.cc:240] Negotiation successful
0619 02:39:34.561127 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":84,"threads_started":1}
I20260619 02:39:34.561342 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/ts-1/wal/instance
I20260619 02:39:34.561501 6085 external_mini_cluster.cc:1428] Setting key 35bfe4b9da997c0a247c8ddd7560adfc
I20260619 02:39:34.562041 6462 heartbeater.cc:347] Connected to a master server at 127.5.241.126:36941
I20260619 02:39:34.562165 6462 heartbeater.cc:464] Registering TS with master...
I20260619 02:39:34.562364 6462 heartbeater.cc:511] Master 127.5.241.126:36941 requested a full tablet report, sending...
I20260619 02:39:34.562867 6137 ts_manager.cc:194] Registered new tserver with Master: ec89db343470413f888ef557a7f44f34 (127.5.241.66:43815)
I20260619 02:39:34.563256 6137 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.66@KRBTEST.COM'} at 127.5.241.66:41503
WARNING: no policy specified for kudu/127.5.241.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab.
Entry for principal kudu/127.5.241.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab.
I20260619 02:39:34.586056 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.67:0
--local_ip_for_outbound_sockets=127.5.241.67
--webserver_interface=127.5.241.67
--webserver_port=0
--tserver_master_addrs=127.5.241.126:36941
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab
--principal=kudu/127.5.241.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:46309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:39:34.671025 6467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:39:34.671236 6467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:39:34.671262 6467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:39:34.672572 6467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:39:34.672618 6467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:39:34.672648 6467 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.67
I20260619 02:39:34.674263 6467 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:46309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab
--principal=kudu/127.5.241.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.241.67
--webserver_port=0
--tserver_master_addrs=127.5.241.126:36941
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6467
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.67
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:39:34.674566 6467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:39:34.674846 6467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:39:34.680473 6475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:39:34.680514 6467 server_base.cc:1034] running on GCE node
W20260619 02:39:34.680481 6472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:34.680483 6473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:39:34.680881 6467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:39:34.681094 6467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:39:34.682237 6467 hybrid_clock.cc:648] HybridClock initialized: now 1781836774682227 us; error 53 us; skew 500 ppm
Jun 19 02:39:34 dist-test-slave-zpbg krb5kdc[6092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836774, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:39:34.684806 6467 init.cc:377] Logged in from keytab as kudu/127.5.241.67@KRBTEST.COM (short username kudu)
I20260619 02:39:34.685290 6467 webserver.cc:460] Webserver started at http://127.5.241.67:38919/ using document root <none> and password file <none>
I20260619 02:39:34.685529 6467 fs_manager.cc:359] Metadata directory not provided
I20260619 02:39:34.685606 6467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:39:34.685755 6467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:39:34.686885 6467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/ts-2/data/instance:
uuid: "200ae496794148b19b1ab0bf9ea56a42"
format_stamp: "Formatted at 2026-06-19 02:39:34 on dist-test-slave-zpbg"
server_key: "67371605ae326586f887ab431636cc28"
server_key_iv: "d4eb6f8353afb496bb6f3793655db5b4"
server_key_version: "encryptionkey@0"
I20260619 02:39:34.687251 6467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/ts-2/wal/instance:
uuid: "200ae496794148b19b1ab0bf9ea56a42"
format_stamp: "Formatted at 2026-06-19 02:39:34 on dist-test-slave-zpbg"
server_key: "67371605ae326586f887ab431636cc28"
server_key_iv: "d4eb6f8353afb496bb6f3793655db5b4"
server_key_version: "encryptionkey@0"
I20260619 02:39:34.688743 6467 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:39:34.689775 6482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:34.690106 6467 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:39:34.690235 6467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/ts-2/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/ts-2/wal
uuid: "200ae496794148b19b1ab0bf9ea56a42"
format_stamp: "Formatted at 2026-06-19 02:39:34 on dist-test-slave-zpbg"
server_key: "67371605ae326586f887ab431636cc28"
server_key_iv: "d4eb6f8353afb496bb6f3793655db5b4"
server_key_version: "encryptionkey@0"
I20260619 02:39:34.690320 6467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:39:34.709218 6467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:39:34.711838 6467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:39:34.711969 6467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:39:34.712163 6467 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:39:34.712502 6467 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:39:34.712543 6467 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:34.712566 6467 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:39:34.712616 6467 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:34.719673 6467 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.67:38193
I20260619 02:39:34.719713 6595 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.67:38193 every 8 connection(s)
I20260619 02:39:34.720072 6467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
I20260619 02:39:34.720968 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 6467
I20260619 02:39:34.721055 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836772852275-6085-0/minicluster-data/ts-2/wal/instance
I20260619 02:39:34.721230 6085 external_mini_cluster.cc:1428] Setting key 4d1d3c2f84184facd2ad81693c1ce602
Jun 19 02:39:34 dist-test-slave-zpbg krb5kdc[6092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836774, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.67@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:39:34.729575 6194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:34.720891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:58575 (local address 127.5.241.126:36941)
0619 02:39:34.720979 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:34.720982 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:34.721119 (+ 137us) server_negotiation.cc:407] Connection header received
0619 02:39:34.721827 (+ 708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:34.721828 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:34.721839 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:34.721883 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:34.723007 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:34.723492 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:34.724498 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:34.724687 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:34.724753 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:34.726873 (+ 2120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:34.726875 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:34.726875 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:34.726881 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:34.728311 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:34.728669 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:34.728670 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:34.728670 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:34.728719 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:34.728998 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:34.728999 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:34.728999 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:34.729101 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:34.729155 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:34.729450 (+ 295us) server_negotiation.cc:299] Negotiation successful
0619 02:39:34.729511 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 02:39:34.729727 6598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:34.720813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:36941 (local address 127.5.241.67:58575)
0619 02:39:34.720978 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:34.721022 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:34.721711 (+ 689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:34.721970 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:34.721974 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:34.722659 (+ 685us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:39:34.722929 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:34.722934 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:34.723587 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:34.723587 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:34.724407 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:34.724408 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:34.724769 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:34.724770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:34.724877 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:34.725369 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:39:34.725377 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:39:34.726752 (+ 1375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:39:34.728383 (+ 1631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:34.728389 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:34.728390 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:34.728606 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:34.728797 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:34.728798 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:34.728798 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:34.728907 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:34.729162 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:39:34.729166 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:39:34.729277 (+ 111us) client_negotiation.cc:769] Sending connection context
0619 02:39:34.729357 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 02:39:34.729534 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":92,"threads_started":1}
I20260619 02:39:34.730263 6596 heartbeater.cc:347] Connected to a master server at 127.5.241.126:36941
I20260619 02:39:34.730352 6596 heartbeater.cc:464] Registering TS with master...
I20260619 02:39:34.730600 6596 heartbeater.cc:511] Master 127.5.241.126:36941 requested a full tablet report, sending...
I20260619 02:39:34.731082 6137 ts_manager.cc:194] Registered new tserver with Master: 200ae496794148b19b1ab0bf9ea56a42 (127.5.241.67:38193)
I20260619 02:39:34.731472 6137 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.67@KRBTEST.COM'} at 127.5.241.67:58575
I20260619 02:39:34.733290 6085 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 02:39:34 dist-test-slave-zpbg krb5kdc[6092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836774, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 02:39:34 dist-test-slave-zpbg krb5kdc[6092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836774, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.241.65@KRBTEST.COM
I20260619 02:39:34.757424 6603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:34.748506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54980 (local address 127.5.241.65:42955)
0619 02:39:34.748688 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:34.748694 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:34.748706 (+ 12us) server_negotiation.cc:407] Connection header received
0619 02:39:34.748739 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:34.748742 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:34.748765 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:34.748848 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:34.749710 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:34.750473 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:34.751387 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:34.751738 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:34.751843 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:34.754300 (+ 2457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:34.754308 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:34.754309 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:34.754317 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:34.756022 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:34.756420 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:34.756430 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:34.756430 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:34.756491 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:34.756701 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:34.756702 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:34.756702 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:34.756937 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:34.757169 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:34.757215 (+ 46us) server_negotiation.cc:299] Negotiation successful
0619 02:39:34.757298 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":82,"threads_started":1}
W20260619 02:39:34.757871 6242 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54980
Jun 19 02:39:34 dist-test-slave-zpbg krb5kdc[6092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 02:39:34 dist-test-slave-zpbg krb5kdc[6092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.241.65@KRBTEST.COM
I20260619 02:39:34.781178 6603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:34.773979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54982 (local address 127.5.241.65:42955)
0619 02:39:34.774063 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:34.774065 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:34.774078 (+ 13us) server_negotiation.cc:407] Connection header received
0619 02:39:34.774152 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:34.774153 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:34.774162 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:34.774218 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:34.774831 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:34.775314 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:34.776121 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:34.776325 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:34.776392 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:34.778695 (+ 2303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:34.778696 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:34.778696 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:34.778704 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:34.780041 (+ 1337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:34.780362 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:34.780362 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:34.780363 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:34.780412 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:34.780622 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:34.780623 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:34.780624 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:34.780755 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:34.780876 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:34.780959 (+ 83us) server_negotiation.cc:299] Negotiation successful
0619 02:39:34.781044 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 02:39:34.781813 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 6199
I20260619 02:39:34.786583 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 6333
I20260619 02:39:34.791780 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 6467
I20260619 02:39:34.796226 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 6105
2026-06-19T02:39:34Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1951 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:39:34 dist-test-slave-zpbg krb5kdc[6610](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:39:34 dist-test-slave-zpbg krb5kdc[6610](info): set up 2 sockets
Jun 19 02:39:34 dist-test-slave-zpbg krb5kdc[6610](info): commencing operation
krb5kdc: starting...
W20260619 02:39:35.854475 6085 mini_kdc.cc:121] Time spent starting KDC: real 1.038s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Jun 19 02:39:35 dist-test-slave-zpbg krb5kdc[6610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836775, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:39:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:39:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.241.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
I20260619 02:39:35.973752 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:40635
--webserver_interface=127.5.241.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:39767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:40635
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:39:36.055236 6623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:39:36.055459 6623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:39:36.055482 6623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:39:36.056608 6623 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:39:36.056648 6623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:39:36.056661 6623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:39:36.056675 6623 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:39:36.056687 6623 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:39:36.058079 6623 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:39767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:40635
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:40635
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.241.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:39:36.058277 6623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:39:36.058447 6623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:39:36.063392 6629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:36.063443 6628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:36.063413 6631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:39:36.063613 6623 server_base.cc:1034] running on GCE node
I20260619 02:39:36.063898 6623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:39:36.064257 6623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:39:36.065415 6623 hybrid_clock.cc:648] HybridClock initialized: now 1781836776065396 us; error 38 us; skew 500 ppm
Jun 19 02:39:36 dist-test-slave-zpbg krb5kdc[6610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836776, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:39:36.068063 6623 init.cc:377] Logged in from keytab as kudu/127.5.241.126@KRBTEST.COM (short username kudu)
I20260619 02:39:36.068601 6623 webserver.cc:460] Webserver started at http://127.5.241.126:37109/ using document root <none> and password file <none>
I20260619 02:39:36.068837 6623 fs_manager.cc:359] Metadata directory not provided
I20260619 02:39:36.068905 6623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:39:36.069056 6623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:39:36.070164 6623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/master-0/data/instance:
uuid: "c5467d8c25534208916635747dbb6240"
format_stamp: "Formatted at 2026-06-19 02:39:36 on dist-test-slave-zpbg"
server_key: "3bc9dd71149646646eb03d7ebd823953"
server_key_iv: "210656bdc2d1f1e6c68042693425a9f6"
server_key_version: "encryptionkey@0"
I20260619 02:39:36.070544 6623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/master-0/wal/instance:
uuid: "c5467d8c25534208916635747dbb6240"
format_stamp: "Formatted at 2026-06-19 02:39:36 on dist-test-slave-zpbg"
server_key: "3bc9dd71149646646eb03d7ebd823953"
server_key_iv: "210656bdc2d1f1e6c68042693425a9f6"
server_key_version: "encryptionkey@0"
I20260619 02:39:36.071962 6623 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:39:36.072750 6638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:36.072916 6623 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:39:36.073016 6623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "c5467d8c25534208916635747dbb6240"
format_stamp: "Formatted at 2026-06-19 02:39:36 on dist-test-slave-zpbg"
server_key: "3bc9dd71149646646eb03d7ebd823953"
server_key_iv: "210656bdc2d1f1e6c68042693425a9f6"
server_key_version: "encryptionkey@0"
I20260619 02:39:36.073103 6623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:39:36.092931 6623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:39:36.094885 6623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:39:36.095058 6623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:39:36.099184 6623 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.126:40635
I20260619 02:39:36.099231 6690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.126:40635 every 8 connection(s)
I20260619 02:39:36.099602 6623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:39:36.100489 6691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:39:36.102718 6691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5467d8c25534208916635747dbb6240: Bootstrap starting.
I20260619 02:39:36.103329 6691 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c5467d8c25534208916635747dbb6240: Neither blocks nor log segments found. Creating new log.
I20260619 02:39:36.103576 6691 log.cc:824] T 00000000000000000000000000000000 P c5467d8c25534208916635747dbb6240: Log is configured to *not* fsync() on all Append() calls
I20260619 02:39:36.104264 6691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5467d8c25534208916635747dbb6240: No bootstrap required, opened a new log
I20260619 02:39:36.105777 6691 raft_consensus.cc:348] T 00000000000000000000000000000000 P c5467d8c25534208916635747dbb6240 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5467d8c25534208916635747dbb6240" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 40635 } }
I20260619 02:39:36.105912 6691 raft_consensus.cc:374] T 00000000000000000000000000000000 P c5467d8c25534208916635747dbb6240 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:39:36.105934 6691 raft_consensus.cc:729] T 00000000000000000000000000000000 P c5467d8c25534208916635747dbb6240 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5467d8c25534208916635747dbb6240, State: Initialized, Role: FOLLOWER
I20260619 02:39:36.106038 6691 consensus_queue.cc:260] T 00000000000000000000000000000000 P c5467d8c25534208916635747dbb6240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5467d8c25534208916635747dbb6240" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 40635 } }
I20260619 02:39:36.106120 6691 raft_consensus.cc:388] T 00000000000000000000000000000000 P c5467d8c25534208916635747dbb6240 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:39:36.106156 6691 raft_consensus.cc:482] T 00000000000000000000000000000000 P c5467d8c25534208916635747dbb6240 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:39:36.106230 6691 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c5467d8c25534208916635747dbb6240 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:39:36.106825 6691 raft_consensus.cc:504] T 00000000000000000000000000000000 P c5467d8c25534208916635747dbb6240 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5467d8c25534208916635747dbb6240" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 40635 } }
I20260619 02:39:36.106957 6691 leader_election.cc:302] T 00000000000000000000000000000000 P c5467d8c25534208916635747dbb6240 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5467d8c25534208916635747dbb6240; no voters:
I20260619 02:39:36.107275 6691 leader_election.cc:288] T 00000000000000000000000000000000 P c5467d8c25534208916635747dbb6240 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:39:36.107339 6694 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c5467d8c25534208916635747dbb6240 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:39:36.107558 6691 sys_catalog.cc:564] T 00000000000000000000000000000000 P c5467d8c25534208916635747dbb6240 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:39:36.107698 6694 raft_consensus.cc:686] T 00000000000000000000000000000000 P c5467d8c25534208916635747dbb6240 [term 1 LEADER]: Becoming Leader. State: Replica: c5467d8c25534208916635747dbb6240, State: Running, Role: LEADER
I20260619 02:39:36.107841 6694 consensus_queue.cc:237] T 00000000000000000000000000000000 P c5467d8c25534208916635747dbb6240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5467d8c25534208916635747dbb6240" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 40635 } }
I20260619 02:39:36.108510 6696 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5467d8c25534208916635747dbb6240 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5467d8c25534208916635747dbb6240. Latest consensus state: current_term: 1 leader_uuid: "c5467d8c25534208916635747dbb6240" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5467d8c25534208916635747dbb6240" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 40635 } } }
I20260619 02:39:36.108623 6696 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5467d8c25534208916635747dbb6240 [sys.catalog]: This master's current role is: LEADER
I20260619 02:39:36.108724 6695 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5467d8c25534208916635747dbb6240 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c5467d8c25534208916635747dbb6240" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5467d8c25534208916635747dbb6240" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 40635 } } }
I20260619 02:39:36.108793 6695 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5467d8c25534208916635747dbb6240 [sys.catalog]: This master's current role is: LEADER
I20260619 02:39:36.108975 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 6623
I20260619 02:39:36.109015 6707 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:39:36.109063 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/master-0/wal/instance
I20260619 02:39:36.109230 6085 external_mini_cluster.cc:1428] Setting key 11e3f75b3ebc6c4e449a175497a81379
I20260619 02:39:36.109508 6707 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:39:36.111138 6707 catalog_manager.cc:1347] Generated new cluster ID: 32f93b0baa9e43079c8114b402ea77f2
I20260619 02:39:36.111240 6707 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 02:39:36 dist-test-slave-zpbg krb5kdc[6610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836775, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:39:36.116942 6707 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:39:36.117497 6707 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:39:36.121654 6713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:36.109852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35540 (local address 127.5.241.126:40635)
0619 02:39:36.110098 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:36.110109 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:36.110140 (+ 31us) server_negotiation.cc:407] Connection header received
0619 02:39:36.110264 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:36.110271 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:36.110335 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:36.110476 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:36.111378 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:36.112736 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:36.113816 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:36.114410 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:36.114510 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:36.117507 (+ 2997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:36.117512 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:36.117512 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:36.117526 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:36.119991 (+ 2465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:36.120412 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:36.120415 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:36.120416 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:36.120525 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:36.120735 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:36.120735 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:36.120736 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:36.121008 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:36.121155 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:36.121265 (+ 110us) server_negotiation.cc:299] Negotiation successful
0619 02:39:36.121412 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":83,"threads_started":1}
I20260619 02:39:36.131457 6707 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c5467d8c25534208916635747dbb6240: Generated new TSK 0
I20260619 02:39:36.131721 6707 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.241.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab.
Entry for principal kudu/127.5.241.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab.
I20260619 02:39:36.163111 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.65:0
--local_ip_for_outbound_sockets=127.5.241.65
--webserver_interface=127.5.241.65
--webserver_port=0
--tserver_master_addrs=127.5.241.126:40635
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab
--principal=kudu/127.5.241.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:39767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:39:36.245398 6717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:39:36.245671 6717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:39:36.245713 6717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:39:36.247001 6717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:39:36.247062 6717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:39:36.247107 6717 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.65
I20260619 02:39:36.248685 6717 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:39767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab
--principal=kudu/127.5.241.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.241.65
--webserver_port=0
--tserver_master_addrs=127.5.241.126:40635
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6717
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.65
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:39:36.248960 6717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:39:36.249199 6717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:39:36.254595 6722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:36.254575 6725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:39:36.254608 6717 server_base.cc:1034] running on GCE node
W20260619 02:39:36.254575 6723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:39:36.255092 6717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:39:36.255309 6717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:39:36.256458 6717 hybrid_clock.cc:648] HybridClock initialized: now 1781836776256435 us; error 42 us; skew 500 ppm
Jun 19 02:39:36 dist-test-slave-zpbg krb5kdc[6610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836776, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:39:36.259363 6717 init.cc:377] Logged in from keytab as kudu/127.5.241.65@KRBTEST.COM (short username kudu)
I20260619 02:39:36.259956 6717 webserver.cc:460] Webserver started at http://127.5.241.65:37963/ using document root <none> and password file <none>
I20260619 02:39:36.260197 6717 fs_manager.cc:359] Metadata directory not provided
I20260619 02:39:36.260246 6717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:39:36.260394 6717 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:39:36.261493 6717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/ts-0/data/instance:
uuid: "4d13a99ad37a48aea06f81e0ef04b8de"
format_stamp: "Formatted at 2026-06-19 02:39:36 on dist-test-slave-zpbg"
server_key: "af4f6fc15a50d23409f1847b8af23900"
server_key_iv: "7973153e4000608b27a3c7a62756677d"
server_key_version: "encryptionkey@0"
I20260619 02:39:36.261901 6717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/ts-0/wal/instance:
uuid: "4d13a99ad37a48aea06f81e0ef04b8de"
format_stamp: "Formatted at 2026-06-19 02:39:36 on dist-test-slave-zpbg"
server_key: "af4f6fc15a50d23409f1847b8af23900"
server_key_iv: "7973153e4000608b27a3c7a62756677d"
server_key_version: "encryptionkey@0"
I20260619 02:39:36.263326 6717 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:39:36.264117 6732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:36.264330 6717 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:39:36.264423 6717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/ts-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/ts-0/wal
uuid: "4d13a99ad37a48aea06f81e0ef04b8de"
format_stamp: "Formatted at 2026-06-19 02:39:36 on dist-test-slave-zpbg"
server_key: "af4f6fc15a50d23409f1847b8af23900"
server_key_iv: "7973153e4000608b27a3c7a62756677d"
server_key_version: "encryptionkey@0"
I20260619 02:39:36.264498 6717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:39:36.279471 6717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:39:36.281747 6717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:39:36.281924 6717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:39:36.282207 6717 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:39:36.282558 6717 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:39:36.282595 6717 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:36.282620 6717 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:39:36.282671 6717 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:36.289126 6717 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.65:33231
I20260619 02:39:36.289197 6845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.65:33231 every 8 connection(s)
I20260619 02:39:36.289544 6717 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
Jun 19 02:39:36 dist-test-slave-zpbg krb5kdc[6610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836776, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.65@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:39:36.298285 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 6717
I20260619 02:39:36.298391 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/ts-0/wal/instance
I20260619 02:39:36.298542 6085 external_mini_cluster.cc:1428] Setting key 856545eb707af81e23dbae51a0d8132a
I20260619 02:39:36.299952 6713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:36.290340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:37065 (local address 127.5.241.126:40635)
0619 02:39:36.290427 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:36.290430 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:36.290630 (+ 200us) server_negotiation.cc:407] Connection header received
0619 02:39:36.291675 (+ 1045us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:36.291676 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:36.291689 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:36.291745 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:36.292410 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:36.292931 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:36.293950 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:36.294164 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:36.294243 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:36.296513 (+ 2270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:36.296515 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:36.296515 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:36.296525 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:36.297973 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:36.298444 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:36.298446 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:36.298446 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:36.298533 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:36.299127 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:36.299128 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:36.299128 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:36.299304 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:36.299431 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:36.299722 (+ 291us) server_negotiation.cc:299] Negotiation successful
0619 02:39:36.299836 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 02:39:36.300151 6848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:36.290300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:40635 (local address 127.5.241.65:37065)
0619 02:39:36.290488 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:36.290525 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:36.291558 (+ 1033us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:36.291828 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:36.291832 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:36.292115 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:39:36.292306 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:36.292311 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:36.293038 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:36.293039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:36.293856 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:36.293858 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:36.294250 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:36.294251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:36.294385 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:36.294991 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:39:36.294999 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:39:36.296409 (+ 1410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:39:36.298061 (+ 1652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:36.298069 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:36.298070 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:36.298317 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:36.298619 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:36.298621 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:36.298621 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:36.298836 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:36.299408 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:39:36.299416 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:39:36.299571 (+ 155us) client_negotiation.cc:769] Sending connection context
0619 02:39:36.299683 (+ 112us) client_negotiation.cc:240] Negotiation successful
0619 02:39:36.299853 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":107,"threads_started":1}
I20260619 02:39:36.301074 6846 heartbeater.cc:347] Connected to a master server at 127.5.241.126:40635
I20260619 02:39:36.301203 6846 heartbeater.cc:464] Registering TS with master...
I20260619 02:39:36.301409 6846 heartbeater.cc:511] Master 127.5.241.126:40635 requested a full tablet report, sending...
I20260619 02:39:36.302044 6655 ts_manager.cc:194] Registered new tserver with Master: 4d13a99ad37a48aea06f81e0ef04b8de (127.5.241.65:33231)
I20260619 02:39:36.302677 6655 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.65@KRBTEST.COM'} at 127.5.241.65:37065
WARNING: no policy specified for kudu/127.5.241.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab.
Entry for principal kudu/127.5.241.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab.
I20260619 02:39:36.325500 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.66:0
--local_ip_for_outbound_sockets=127.5.241.66
--webserver_interface=127.5.241.66
--webserver_port=0
--tserver_master_addrs=127.5.241.126:40635
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab
--principal=kudu/127.5.241.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:39767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:39:36.405897 6851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:39:36.406143 6851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:39:36.406183 6851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:39:36.407385 6851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:39:36.407459 6851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:39:36.407505 6851 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.66
I20260619 02:39:36.409240 6851 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:39767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab
--principal=kudu/127.5.241.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.241.66
--webserver_port=0
--tserver_master_addrs=127.5.241.126:40635
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.66
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:39:36.409505 6851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:39:36.409755 6851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:39:36.414698 6857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:36.414705 6856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:36.414705 6859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:39:36.414878 6851 server_base.cc:1034] running on GCE node
I20260619 02:39:36.415207 6851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:39:36.415442 6851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:39:36.416568 6851 hybrid_clock.cc:648] HybridClock initialized: now 1781836776416546 us; error 39 us; skew 500 ppm
Jun 19 02:39:36 dist-test-slave-zpbg krb5kdc[6610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836776, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:39:36.419215 6851 init.cc:377] Logged in from keytab as kudu/127.5.241.66@KRBTEST.COM (short username kudu)
I20260619 02:39:36.419677 6851 webserver.cc:460] Webserver started at http://127.5.241.66:42707/ using document root <none> and password file <none>
I20260619 02:39:36.419919 6851 fs_manager.cc:359] Metadata directory not provided
I20260619 02:39:36.419987 6851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:39:36.420131 6851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:39:36.421188 6851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/ts-1/data/instance:
uuid: "38d8e654a51b4d08be7882f4d4730dce"
format_stamp: "Formatted at 2026-06-19 02:39:36 on dist-test-slave-zpbg"
server_key: "e5eec0b9b8dd5f197099c4e370593ad5"
server_key_iv: "f98706ec3daad7eef9fc5eef4350fdbf"
server_key_version: "encryptionkey@0"
I20260619 02:39:36.421555 6851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/ts-1/wal/instance:
uuid: "38d8e654a51b4d08be7882f4d4730dce"
format_stamp: "Formatted at 2026-06-19 02:39:36 on dist-test-slave-zpbg"
server_key: "e5eec0b9b8dd5f197099c4e370593ad5"
server_key_iv: "f98706ec3daad7eef9fc5eef4350fdbf"
server_key_version: "encryptionkey@0"
I20260619 02:39:36.422951 6851 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 02:39:36.423781 6866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:36.423978 6851 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:39:36.424077 6851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/ts-1/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/ts-1/wal
uuid: "38d8e654a51b4d08be7882f4d4730dce"
format_stamp: "Formatted at 2026-06-19 02:39:36 on dist-test-slave-zpbg"
server_key: "e5eec0b9b8dd5f197099c4e370593ad5"
server_key_iv: "f98706ec3daad7eef9fc5eef4350fdbf"
server_key_version: "encryptionkey@0"
I20260619 02:39:36.424162 6851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:39:36.444661 6851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:39:36.447141 6851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:39:36.447304 6851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:39:36.447542 6851 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:39:36.447853 6851 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:39:36.447906 6851 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:36.447947 6851 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:39:36.447974 6851 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:36.453727 6851 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.66:39145
I20260619 02:39:36.453775 6979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.66:39145 every 8 connection(s)
I20260619 02:39:36.454113 6851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
I20260619 02:39:36.460819 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 6851
I20260619 02:39:36.460915 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/ts-1/wal/instance
I20260619 02:39:36.461082 6085 external_mini_cluster.cc:1428] Setting key cfc4ea9392f775335ab3eec95a7310ff
Jun 19 02:39:36 dist-test-slave-zpbg krb5kdc[6610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836776, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.66@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:39:36.465672 6713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:36.454939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:48929 (local address 127.5.241.126:40635)
0619 02:39:36.454986 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:36.454988 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:36.455738 (+ 750us) server_negotiation.cc:407] Connection header received
0619 02:39:36.456860 (+ 1122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:36.456861 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:36.456876 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:36.456912 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:36.457875 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:36.458455 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:36.459581 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:36.459793 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:36.459897 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:36.462347 (+ 2450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:36.462348 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:36.462349 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:36.462356 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:36.464101 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:36.464605 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:36.464607 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:36.464607 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:36.464672 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:36.464956 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:36.464957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:36.464957 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:36.465078 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:36.465159 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:36.465538 (+ 379us) server_negotiation.cc:299] Negotiation successful
0619 02:39:36.465607 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 02:39:36.465920 6982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:36.454929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:40635 (local address 127.5.241.66:48929)
0619 02:39:36.455574 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:36.455623 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:36.456722 (+ 1099us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:36.457109 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:36.457114 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:36.457473 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:39:36.457762 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:36.457769 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:36.458560 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:36.458561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:36.459487 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:36.459490 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:36.459902 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:36.459903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:36.460042 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:36.460593 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:39:36.460601 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:39:36.462251 (+ 1650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:39:36.464196 (+ 1945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:36.464203 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:36.464205 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:36.464522 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:36.464744 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:36.464745 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:36.464746 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:36.464892 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:36.465153 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:39:36.465159 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:39:36.465333 (+ 174us) client_negotiation.cc:769] Sending connection context
0619 02:39:36.465458 (+ 125us) client_negotiation.cc:240] Negotiation successful
0619 02:39:36.465646 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":85,"threads_started":1}
I20260619 02:39:36.466456 6980 heartbeater.cc:347] Connected to a master server at 127.5.241.126:40635
I20260619 02:39:36.466570 6980 heartbeater.cc:464] Registering TS with master...
I20260619 02:39:36.466801 6980 heartbeater.cc:511] Master 127.5.241.126:40635 requested a full tablet report, sending...
I20260619 02:39:36.467254 6655 ts_manager.cc:194] Registered new tserver with Master: 38d8e654a51b4d08be7882f4d4730dce (127.5.241.66:39145)
I20260619 02:39:36.467660 6655 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.66@KRBTEST.COM'} at 127.5.241.66:48929
WARNING: no policy specified for kudu/127.5.241.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab.
Entry for principal kudu/127.5.241.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab.
I20260619 02:39:36.489400 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.67:0
--local_ip_for_outbound_sockets=127.5.241.67
--webserver_interface=127.5.241.67
--webserver_port=0
--tserver_master_addrs=127.5.241.126:40635
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab
--principal=kudu/127.5.241.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:39767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:39:36.569640 6985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:39:36.569842 6985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:39:36.569866 6985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:39:36.571063 6985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:39:36.571102 6985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:39:36.571130 6985 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.67
I20260619 02:39:36.572521 6985 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:39767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab
--principal=kudu/127.5.241.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.241.67
--webserver_port=0
--tserver_master_addrs=127.5.241.126:40635
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6985
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.67
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:39:36.572754 6985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:39:36.572956 6985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:39:36.577818 6991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:36.577822 6993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:36.577816 6990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:39:36.577834 6985 server_base.cc:1034] running on GCE node
I20260619 02:39:36.578269 6985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:39:36.578469 6985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:39:36.579594 6985 hybrid_clock.cc:648] HybridClock initialized: now 1781836776579576 us; error 33 us; skew 500 ppm
Jun 19 02:39:36 dist-test-slave-zpbg krb5kdc[6610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836776, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:39:36.582032 6985 init.cc:377] Logged in from keytab as kudu/127.5.241.67@KRBTEST.COM (short username kudu)
I20260619 02:39:36.582484 6985 webserver.cc:460] Webserver started at http://127.5.241.67:44857/ using document root <none> and password file <none>
I20260619 02:39:36.582679 6985 fs_manager.cc:359] Metadata directory not provided
I20260619 02:39:36.582723 6985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:39:36.582866 6985 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:39:36.583868 6985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/ts-2/data/instance:
uuid: "dc98c5647a5946889a013eae5a6b5581"
format_stamp: "Formatted at 2026-06-19 02:39:36 on dist-test-slave-zpbg"
server_key: "a3a6e523e62f0498d75844058238dff0"
server_key_iv: "7a2ee7ea88bd8d73dece620b86569d35"
server_key_version: "encryptionkey@0"
I20260619 02:39:36.584165 6985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/ts-2/wal/instance:
uuid: "dc98c5647a5946889a013eae5a6b5581"
format_stamp: "Formatted at 2026-06-19 02:39:36 on dist-test-slave-zpbg"
server_key: "a3a6e523e62f0498d75844058238dff0"
server_key_iv: "7a2ee7ea88bd8d73dece620b86569d35"
server_key_version: "encryptionkey@0"
I20260619 02:39:36.585431 6985 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 02:39:36.586122 7000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:36.586309 6985 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:39:36.586395 6985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/ts-2/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/ts-2/wal
uuid: "dc98c5647a5946889a013eae5a6b5581"
format_stamp: "Formatted at 2026-06-19 02:39:36 on dist-test-slave-zpbg"
server_key: "a3a6e523e62f0498d75844058238dff0"
server_key_iv: "7a2ee7ea88bd8d73dece620b86569d35"
server_key_version: "encryptionkey@0"
I20260619 02:39:36.586467 6985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:39:36.607005 6985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:39:36.609516 6985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:39:36.609663 6985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:39:36.609895 6985 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:39:36.610217 6985 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:39:36.610250 6985 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:36.610272 6985 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:39:36.610312 6985 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:36.615955 6985 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.67:43757
I20260619 02:39:36.616029 7113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.67:43757 every 8 connection(s)
I20260619 02:39:36.616418 6985 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
Jun 19 02:39:36 dist-test-slave-zpbg krb5kdc[6610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836776, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.67@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:39:36.624511 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 6985
I20260619 02:39:36.624631 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836772852275-6085-0/minicluster-data/ts-2/wal/instance
I20260619 02:39:36.624807 6085 external_mini_cluster.cc:1428] Setting key 898ccf09cc052eb2fd726e2fa812f5da
I20260619 02:39:36.626075 6713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:36.617138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:34779 (local address 127.5.241.126:40635)
0619 02:39:36.617184 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:36.617186 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:36.617398 (+ 212us) server_negotiation.cc:407] Connection header received
0619 02:39:36.618363 (+ 965us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:36.618365 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:36.618377 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:36.618444 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:36.619144 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:36.619645 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:36.620664 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:36.620852 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:36.620928 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:36.623099 (+ 2171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:36.623101 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:36.623101 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:36.623109 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:36.624440 (+ 1331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:36.625058 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:36.625059 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:36.625059 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:36.625144 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:36.625408 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:36.625409 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:36.625410 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:36.625534 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:36.625612 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:36.625904 (+ 292us) server_negotiation.cc:299] Negotiation successful
0619 02:39:36.625977 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:39:36.626202 7116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:36.617121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:40635 (local address 127.5.241.67:34779)
0619 02:39:36.617285 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:36.617319 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:36.618251 (+ 932us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:36.618513 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:36.618517 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:36.618841 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:39:36.619040 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:36.619045 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:36.619745 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:36.619747 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:36.620554 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:36.620556 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:36.620932 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:36.620933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:36.621062 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:36.621571 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:39:36.621579 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:39:36.623006 (+ 1427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:39:36.624575 (+ 1569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:36.624581 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:36.624584 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:36.624903 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:36.625221 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:36.625222 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:36.625223 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:36.625350 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:36.625601 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:39:36.625608 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:39:36.625758 (+ 150us) client_negotiation.cc:769] Sending connection context
0619 02:39:36.625873 (+ 115us) client_negotiation.cc:240] Negotiation successful
0619 02:39:36.626010 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":93,"threads_started":1}
I20260619 02:39:36.626649 7114 heartbeater.cc:347] Connected to a master server at 127.5.241.126:40635
I20260619 02:39:36.626812 7114 heartbeater.cc:464] Registering TS with master...
I20260619 02:39:36.626988 7114 heartbeater.cc:511] Master 127.5.241.126:40635 requested a full tablet report, sending...
I20260619 02:39:36.627352 6655 ts_manager.cc:194] Registered new tserver with Master: dc98c5647a5946889a013eae5a6b5581 (127.5.241.67:43757)
I20260619 02:39:36.627730 6655 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.67@KRBTEST.COM'} at 127.5.241.67:34779
I20260619 02:39:36.636833 6085 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:39:36.644743 6713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:36.638520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35548 (local address 127.5.241.126:40635)
0619 02:39:36.638600 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:36.638602 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:36.638687 (+ 85us) server_negotiation.cc:407] Connection header received
0619 02:39:36.638816 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:36.638817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:36.638828 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:36.638912 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:36.639515 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:36.640012 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:36.640736 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:36.640934 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:36.641021 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:36.642244 (+ 1223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:36.642246 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:36.642246 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:36.642253 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:36.643540 (+ 1287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:36.643904 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:36.643905 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:36.643905 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:36.643965 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:36.644178 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:36.644179 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:36.644179 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:36.644332 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:36.644443 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:36.644562 (+ 119us) server_negotiation.cc:299] Negotiation successful
0619 02:39:36.644642 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 02:39:36.647202 6655 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35548:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:39:36.648813 6655 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:39:36.655676 7125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:36.651712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:33231 (local address 127.0.0.1:41416)
0619 02:39:36.651867 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:36.651880 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:36.651935 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:36.652256 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:36.652257 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:36.652260 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:39:36.652310 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:36.652312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:36.653764 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:36.653765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:36.654642 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:36.654644 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:36.655357 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:36.655359 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:36.655434 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:36.655448 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 02:39:36.655547 (+ 99us) client_negotiation.cc:240] Negotiation successful
0619 02:39:36.655601 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":69,"threads_started":1}
I20260619 02:39:36.655684 7130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:36.651937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41416 (local address 127.5.241.65:33231)
0619 02:39:36.652076 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:36.652082 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:36.652092 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:39:36.652134 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:36.652136 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:36.652153 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:36.652385 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:39:36.652462 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:36.653651 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:36.654726 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:36.655258 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:36.655355 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:36.655417 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:36.655564 (+ 147us) server_negotiation.cc:299] Negotiation successful
0619 02:39:36.655606 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":72,"threads_started":1}
I20260619 02:39:36.655872 7126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:36.651525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.66:39145 (local address 127.0.0.1:36612)
0619 02:39:36.651685 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:36.651727 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:36.651794 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:36.652298 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:36.652299 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:36.652301 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:39:36.652339 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:36.652340 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:36.653631 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:36.653632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:36.654698 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:36.654700 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:36.655616 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:36.655618 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:36.655683 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:36.655687 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:39:36.655717 (+ 30us) client_negotiation.cc:240] Negotiation successful
0619 02:39:36.655770 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":2,"thread_start_us":45,"threads_started":1}
I20260619 02:39:36.656216 7128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:36.651623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36612 (local address 127.5.241.66:39145)
0619 02:39:36.652130 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:36.652134 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:36.652142 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:39:36.652179 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:36.652181 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:36.652204 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:36.652421 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:39:36.652499 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:36.653533 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:36.654807 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:36.655514 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:36.655991 (+ 477us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:36.656076 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:36.656104 (+ 28us) server_negotiation.cc:299] Negotiation successful
0619 02:39:36.656145 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":472,"thread_start_us":112,"threads_started":1}
I20260619 02:39:36.656493 6780 tablet_service.cc:1467] Processing CreateTablet for tablet 1ac7a2a46eb9444d904fcce518fa034c (DEFAULT_TABLE table=test-table [id=c5d7f79e01d74873a8aa25103f8c14b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:39:36.656850 6780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ac7a2a46eb9444d904fcce518fa034c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:39:36.656886 6914 tablet_service.cc:1467] Processing CreateTablet for tablet 1ac7a2a46eb9444d904fcce518fa034c (DEFAULT_TABLE table=test-table [id=c5d7f79e01d74873a8aa25103f8c14b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:39:36.657239 6914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ac7a2a46eb9444d904fcce518fa034c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:39:36.658936 7129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:36.651529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.67:43757 (local address 127.0.0.1:53216)
0619 02:39:36.651820 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:36.651830 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:36.651869 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:36.652124 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:36.652128 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:36.652131 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:39:36.652205 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:36.652207 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:36.653281 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:36.653283 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:36.654671 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:36.654674 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:36.655670 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:36.655671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:36.655882 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:36.655886 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:39:36.656012 (+ 126us) client_negotiation.cc:240] Negotiation successful
0619 02:39:36.658852 (+ 2840us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":79,"threads_started":1}
I20260619 02:39:36.659179 7127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:36.651620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53216 (local address 127.5.241.67:43757)
0619 02:39:36.651947 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:36.651952 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:36.651962 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:39:36.652010 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:36.652012 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:36.652025 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:36.652116 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:39:36.652298 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:36.653177 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:36.654798 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:36.655522 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:36.655586 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:36.655642 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:36.659053 (+ 3411us) server_negotiation.cc:299] Negotiation successful
0619 02:39:36.659108 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":74,"threads_started":1}
I20260619 02:39:36.659827 7048 tablet_service.cc:1467] Processing CreateTablet for tablet 1ac7a2a46eb9444d904fcce518fa034c (DEFAULT_TABLE table=test-table [id=c5d7f79e01d74873a8aa25103f8c14b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:39:36.660100 7131 tablet_bootstrap.cc:492] T 1ac7a2a46eb9444d904fcce518fa034c P 4d13a99ad37a48aea06f81e0ef04b8de: Bootstrap starting.
I20260619 02:39:36.660197 7048 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ac7a2a46eb9444d904fcce518fa034c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:39:36.660329 7132 tablet_bootstrap.cc:492] T 1ac7a2a46eb9444d904fcce518fa034c P 38d8e654a51b4d08be7882f4d4730dce: Bootstrap starting.
I20260619 02:39:36.660706 7131 tablet_bootstrap.cc:654] T 1ac7a2a46eb9444d904fcce518fa034c P 4d13a99ad37a48aea06f81e0ef04b8de: Neither blocks nor log segments found. Creating new log.
I20260619 02:39:36.660945 7132 tablet_bootstrap.cc:654] T 1ac7a2a46eb9444d904fcce518fa034c P 38d8e654a51b4d08be7882f4d4730dce: Neither blocks nor log segments found. Creating new log.
I20260619 02:39:36.661118 7131 log.cc:824] T 1ac7a2a46eb9444d904fcce518fa034c P 4d13a99ad37a48aea06f81e0ef04b8de: Log is configured to *not* fsync() on all Append() calls
I20260619 02:39:36.661474 7132 log.cc:824] T 1ac7a2a46eb9444d904fcce518fa034c P 38d8e654a51b4d08be7882f4d4730dce: Log is configured to *not* fsync() on all Append() calls
I20260619 02:39:36.661808 7131 tablet_bootstrap.cc:492] T 1ac7a2a46eb9444d904fcce518fa034c P 4d13a99ad37a48aea06f81e0ef04b8de: No bootstrap required, opened a new log
I20260619 02:39:36.661875 7131 ts_tablet_manager.cc:1397] T 1ac7a2a46eb9444d904fcce518fa034c P 4d13a99ad37a48aea06f81e0ef04b8de: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:39:36.662236 7132 tablet_bootstrap.cc:492] T 1ac7a2a46eb9444d904fcce518fa034c P 38d8e654a51b4d08be7882f4d4730dce: No bootstrap required, opened a new log
I20260619 02:39:36.662292 7132 ts_tablet_manager.cc:1397] T 1ac7a2a46eb9444d904fcce518fa034c P 38d8e654a51b4d08be7882f4d4730dce: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:39:36.663456 7131 raft_consensus.cc:348] T 1ac7a2a46eb9444d904fcce518fa034c P 4d13a99ad37a48aea06f81e0ef04b8de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38d8e654a51b4d08be7882f4d4730dce" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 39145 } } peers { permanent_uuid: "dc98c5647a5946889a013eae5a6b5581" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 43757 } } peers { permanent_uuid: "4d13a99ad37a48aea06f81e0ef04b8de" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 33231 } }
I20260619 02:39:36.663601 7131 raft_consensus.cc:374] T 1ac7a2a46eb9444d904fcce518fa034c P 4d13a99ad37a48aea06f81e0ef04b8de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:39:36.663647 7131 raft_consensus.cc:729] T 1ac7a2a46eb9444d904fcce518fa034c P 4d13a99ad37a48aea06f81e0ef04b8de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d13a99ad37a48aea06f81e0ef04b8de, State: Initialized, Role: FOLLOWER
I20260619 02:39:36.663802 7131 consensus_queue.cc:260] T 1ac7a2a46eb9444d904fcce518fa034c P 4d13a99ad37a48aea06f81e0ef04b8de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38d8e654a51b4d08be7882f4d4730dce" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 39145 } } peers { permanent_uuid: "dc98c5647a5946889a013eae5a6b5581" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 43757 } } peers { permanent_uuid: "4d13a99ad37a48aea06f81e0ef04b8de" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 33231 } }
I20260619 02:39:36.664067 7131 ts_tablet_manager.cc:1428] T 1ac7a2a46eb9444d904fcce518fa034c P 4d13a99ad37a48aea06f81e0ef04b8de: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:39:36.664212 6846 heartbeater.cc:503] Master 127.5.241.126:40635 was elected leader, sending a full tablet report...
I20260619 02:39:36.664520 7136 tablet_bootstrap.cc:492] T 1ac7a2a46eb9444d904fcce518fa034c P dc98c5647a5946889a013eae5a6b5581: Bootstrap starting.
I20260619 02:39:36.664971 7132 raft_consensus.cc:348] T 1ac7a2a46eb9444d904fcce518fa034c P 38d8e654a51b4d08be7882f4d4730dce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38d8e654a51b4d08be7882f4d4730dce" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 39145 } } peers { permanent_uuid: "dc98c5647a5946889a013eae5a6b5581" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 43757 } } peers { permanent_uuid: "4d13a99ad37a48aea06f81e0ef04b8de" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 33231 } }
I20260619 02:39:36.665150 7132 raft_consensus.cc:374] T 1ac7a2a46eb9444d904fcce518fa034c P 38d8e654a51b4d08be7882f4d4730dce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:39:36.665117 7136 tablet_bootstrap.cc:654] T 1ac7a2a46eb9444d904fcce518fa034c P dc98c5647a5946889a013eae5a6b5581: Neither blocks nor log segments found. Creating new log.
I20260619 02:39:36.665198 7132 raft_consensus.cc:729] T 1ac7a2a46eb9444d904fcce518fa034c P 38d8e654a51b4d08be7882f4d4730dce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38d8e654a51b4d08be7882f4d4730dce, State: Initialized, Role: FOLLOWER
I20260619 02:39:36.665306 7132 consensus_queue.cc:260] T 1ac7a2a46eb9444d904fcce518fa034c P 38d8e654a51b4d08be7882f4d4730dce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38d8e654a51b4d08be7882f4d4730dce" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 39145 } } peers { permanent_uuid: "dc98c5647a5946889a013eae5a6b5581" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 43757 } } peers { permanent_uuid: "4d13a99ad37a48aea06f81e0ef04b8de" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 33231 } }
I20260619 02:39:36.665603 6980 heartbeater.cc:503] Master 127.5.241.126:40635 was elected leader, sending a full tablet report...
I20260619 02:39:36.665607 7132 ts_tablet_manager.cc:1428] T 1ac7a2a46eb9444d904fcce518fa034c P 38d8e654a51b4d08be7882f4d4730dce: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 02:39:36.665836 7136 log.cc:824] T 1ac7a2a46eb9444d904fcce518fa034c P dc98c5647a5946889a013eae5a6b5581: Log is configured to *not* fsync() on all Append() calls
I20260619 02:39:36.666659 7136 tablet_bootstrap.cc:492] T 1ac7a2a46eb9444d904fcce518fa034c P dc98c5647a5946889a013eae5a6b5581: No bootstrap required, opened a new log
I20260619 02:39:36.666744 7136 ts_tablet_manager.cc:1397] T 1ac7a2a46eb9444d904fcce518fa034c P dc98c5647a5946889a013eae5a6b5581: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:39:36.668442 7136 raft_consensus.cc:348] T 1ac7a2a46eb9444d904fcce518fa034c P dc98c5647a5946889a013eae5a6b5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38d8e654a51b4d08be7882f4d4730dce" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 39145 } } peers { permanent_uuid: "dc98c5647a5946889a013eae5a6b5581" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 43757 } } peers { permanent_uuid: "4d13a99ad37a48aea06f81e0ef04b8de" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 33231 } }
I20260619 02:39:36.668582 7136 raft_consensus.cc:374] T 1ac7a2a46eb9444d904fcce518fa034c P dc98c5647a5946889a013eae5a6b5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:39:36.668607 7136 raft_consensus.cc:729] T 1ac7a2a46eb9444d904fcce518fa034c P dc98c5647a5946889a013eae5a6b5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc98c5647a5946889a013eae5a6b5581, State: Initialized, Role: FOLLOWER
I20260619 02:39:36.668746 7136 consensus_queue.cc:260] T 1ac7a2a46eb9444d904fcce518fa034c P dc98c5647a5946889a013eae5a6b5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38d8e654a51b4d08be7882f4d4730dce" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 39145 } } peers { permanent_uuid: "dc98c5647a5946889a013eae5a6b5581" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 43757 } } peers { permanent_uuid: "4d13a99ad37a48aea06f81e0ef04b8de" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 33231 } }
I20260619 02:39:36.669051 7136 ts_tablet_manager.cc:1428] T 1ac7a2a46eb9444d904fcce518fa034c P dc98c5647a5946889a013eae5a6b5581: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 02:39:36.669063 7114 heartbeater.cc:503] Master 127.5.241.126:40635 was elected leader, sending a full tablet report...
W20260619 02:39:36.704694 6981 tablet.cc:2369] T 1ac7a2a46eb9444d904fcce518fa034c P 38d8e654a51b4d08be7882f4d4730dce: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:39:36.738423 7139 raft_consensus.cc:482] T 1ac7a2a46eb9444d904fcce518fa034c P dc98c5647a5946889a013eae5a6b5581 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:39:36.738564 7139 raft_consensus.cc:504] T 1ac7a2a46eb9444d904fcce518fa034c P dc98c5647a5946889a013eae5a6b5581 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38d8e654a51b4d08be7882f4d4730dce" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 39145 } } peers { permanent_uuid: "dc98c5647a5946889a013eae5a6b5581" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 43757 } } peers { permanent_uuid: "4d13a99ad37a48aea06f81e0ef04b8de" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 33231 } }
I20260619 02:39:36.739034 7139 leader_election.cc:288] T 1ac7a2a46eb9444d904fcce518fa034c P dc98c5647a5946889a013eae5a6b5581 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38d8e654a51b4d08be7882f4d4730dce (127.5.241.66:39145), 4d13a99ad37a48aea06f81e0ef04b8de (127.5.241.65:33231)
I20260619 02:39:36.743052 7128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:36.739314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:49849 (local address 127.5.241.66:39145)
0619 02:39:36.739360 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:36.739362 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:36.739381 (+ 19us) server_negotiation.cc:407] Connection header received
0619 02:39:36.739456 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:36.739456 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:36.739470 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:36.739530 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:39:36.739709 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:36.740489 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:36.741993 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:36.742584 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:36.742666 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:36.742675 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:36.742935 (+ 260us) server_negotiation.cc:299] Negotiation successful
0619 02:39:36.742978 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 02:39:36.743052 7130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:36.739446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:42349 (local address 127.5.241.65:33231)
0619 02:39:36.739502 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:36.739504 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:36.739609 (+ 105us) server_negotiation.cc:407] Connection header received
0619 02:39:36.739647 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:36.739648 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:36.739655 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:36.739722 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:39:36.739871 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:36.740572 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:36.741993 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:36.742591 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:36.742666 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:36.742675 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:36.742935 (+ 260us) server_negotiation.cc:299] Negotiation successful
0619 02:39:36.742978 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 02:39:36.743115 7116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:36.739220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.66:39145 (local address 127.5.241.67:49849)
0619 02:39:36.739296 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:36.739312 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:36.739383 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:36.739544 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:36.739546 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:36.739548 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:39:36.739628 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:36.739630 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:36.740630 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:36.740631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:36.741876 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:36.741879 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:36.742720 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:36.742721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:36.742837 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:36.742841 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:39:36.742918 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 02:39:36.742976 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":13}
I20260619 02:39:36.743126 7140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:36.739343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:33231 (local address 127.5.241.67:42349)
0619 02:39:36.739515 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:36.739525 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:36.739602 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:36.739738 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:36.739739 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:36.739740 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:39:36.739804 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:36.739805 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:36.740657 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:36.740657 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:36.741876 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:36.741879 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:36.742720 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:36.742721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:36.742837 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:36.742841 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:39:36.742923 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 02:39:36.742975 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":26,"thread_start_us":107,"threads_started":1}
I20260619 02:39:36.743548 6934 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ac7a2a46eb9444d904fcce518fa034c" candidate_uuid: "dc98c5647a5946889a013eae5a6b5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38d8e654a51b4d08be7882f4d4730dce" is_pre_election: true
I20260619 02:39:36.743542 6800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ac7a2a46eb9444d904fcce518fa034c" candidate_uuid: "dc98c5647a5946889a013eae5a6b5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d13a99ad37a48aea06f81e0ef04b8de" is_pre_election: true
I20260619 02:39:36.743707 6934 raft_consensus.cc:2454] T 1ac7a2a46eb9444d904fcce518fa034c P 38d8e654a51b4d08be7882f4d4730dce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc98c5647a5946889a013eae5a6b5581 in term 0.
I20260619 02:39:36.743707 6800 raft_consensus.cc:2454] T 1ac7a2a46eb9444d904fcce518fa034c P 4d13a99ad37a48aea06f81e0ef04b8de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc98c5647a5946889a013eae5a6b5581 in term 0.
I20260619 02:39:36.743965 7003 leader_election.cc:302] T 1ac7a2a46eb9444d904fcce518fa034c P dc98c5647a5946889a013eae5a6b5581 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38d8e654a51b4d08be7882f4d4730dce, dc98c5647a5946889a013eae5a6b5581; no voters:
I20260619 02:39:36.744161 7139 raft_consensus.cc:2781] T 1ac7a2a46eb9444d904fcce518fa034c P dc98c5647a5946889a013eae5a6b5581 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:39:36.744256 7139 raft_consensus.cc:482] T 1ac7a2a46eb9444d904fcce518fa034c P dc98c5647a5946889a013eae5a6b5581 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:39:36.744277 7139 raft_consensus.cc:3037] T 1ac7a2a46eb9444d904fcce518fa034c P dc98c5647a5946889a013eae5a6b5581 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:39:36.745013 7139 raft_consensus.cc:504] T 1ac7a2a46eb9444d904fcce518fa034c P dc98c5647a5946889a013eae5a6b5581 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38d8e654a51b4d08be7882f4d4730dce" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 39145 } } peers { permanent_uuid: "dc98c5647a5946889a013eae5a6b5581" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 43757 } } peers { permanent_uuid: "4d13a99ad37a48aea06f81e0ef04b8de" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 33231 } }
I20260619 02:39:36.745121 7139 leader_election.cc:288] T 1ac7a2a46eb9444d904fcce518fa034c P dc98c5647a5946889a013eae5a6b5581 [CANDIDATE]: Term 1 election: Requested vote from peers 38d8e654a51b4d08be7882f4d4730dce (127.5.241.66:39145), 4d13a99ad37a48aea06f81e0ef04b8de (127.5.241.65:33231)
I20260619 02:39:36.745337 6934 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ac7a2a46eb9444d904fcce518fa034c" candidate_uuid: "dc98c5647a5946889a013eae5a6b5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38d8e654a51b4d08be7882f4d4730dce"
I20260619 02:39:36.745364 6800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ac7a2a46eb9444d904fcce518fa034c" candidate_uuid: "dc98c5647a5946889a013eae5a6b5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d13a99ad37a48aea06f81e0ef04b8de"
I20260619 02:39:36.745435 6934 raft_consensus.cc:3037] T 1ac7a2a46eb9444d904fcce518fa034c P 38d8e654a51b4d08be7882f4d4730dce [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:39:36.745435 6800 raft_consensus.cc:3037] T 1ac7a2a46eb9444d904fcce518fa034c P 4d13a99ad37a48aea06f81e0ef04b8de [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:39:36.746805 6800 raft_consensus.cc:2454] T 1ac7a2a46eb9444d904fcce518fa034c P 4d13a99ad37a48aea06f81e0ef04b8de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc98c5647a5946889a013eae5a6b5581 in term 1.
I20260619 02:39:36.746802 6934 raft_consensus.cc:2454] T 1ac7a2a46eb9444d904fcce518fa034c P 38d8e654a51b4d08be7882f4d4730dce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc98c5647a5946889a013eae5a6b5581 in term 1.
I20260619 02:39:36.746989 7003 leader_election.cc:302] T 1ac7a2a46eb9444d904fcce518fa034c P dc98c5647a5946889a013eae5a6b5581 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38d8e654a51b4d08be7882f4d4730dce, dc98c5647a5946889a013eae5a6b5581; no voters:
I20260619 02:39:36.747138 7139 raft_consensus.cc:2781] T 1ac7a2a46eb9444d904fcce518fa034c P dc98c5647a5946889a013eae5a6b5581 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:39:36.747283 7139 raft_consensus.cc:686] T 1ac7a2a46eb9444d904fcce518fa034c P dc98c5647a5946889a013eae5a6b5581 [term 1 LEADER]: Becoming Leader. State: Replica: dc98c5647a5946889a013eae5a6b5581, State: Running, Role: LEADER
I20260619 02:39:36.747612 7139 consensus_queue.cc:237] T 1ac7a2a46eb9444d904fcce518fa034c P dc98c5647a5946889a013eae5a6b5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38d8e654a51b4d08be7882f4d4730dce" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 39145 } } peers { permanent_uuid: "dc98c5647a5946889a013eae5a6b5581" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 43757 } } peers { permanent_uuid: "4d13a99ad37a48aea06f81e0ef04b8de" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 33231 } }
I20260619 02:39:36.748553 6654 catalog_manager.cc:5495] T 1ac7a2a46eb9444d904fcce518fa034c P dc98c5647a5946889a013eae5a6b5581 reported cstate change: term changed from 0 to 1, leader changed from <none> to dc98c5647a5946889a013eae5a6b5581 (127.5.241.67). New cstate: current_term: 1 leader_uuid: "dc98c5647a5946889a013eae5a6b5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38d8e654a51b4d08be7882f4d4730dce" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 39145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc98c5647a5946889a013eae5a6b5581" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 43757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d13a99ad37a48aea06f81e0ef04b8de" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 33231 } health_report { overall_health: UNKNOWN } } }
Jun 19 02:39:36 dist-test-slave-zpbg krb5kdc[6610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836775, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.241.65@KRBTEST.COM
I20260619 02:39:36.776322 7130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:36.768125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41428 (local address 127.5.241.65:33231)
0619 02:39:36.768220 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:36.768223 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:36.768337 (+ 114us) server_negotiation.cc:407] Connection header received
0619 02:39:36.768451 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:36.768452 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:36.768467 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:36.768543 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:36.769189 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:36.769682 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:36.770493 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:36.770697 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:36.770823 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:36.773314 (+ 2491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:36.773335 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:36.773335 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:36.773350 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:36.775055 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:36.775484 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:36.775487 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:36.775487 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:36.775557 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:36.775745 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:36.775745 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:36.775746 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:36.775888 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:36.776000 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:36.776091 (+ 91us) server_negotiation.cc:299] Negotiation successful
0619 02:39:36.776178 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
Jun 19 02:39:36 dist-test-slave-zpbg krb5kdc[6610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836776, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
W20260619 02:39:36.790308 6847 tablet.cc:2369] T 1ac7a2a46eb9444d904fcce518fa034c P 4d13a99ad37a48aea06f81e0ef04b8de: Can't schedule compaction. Clean time has not been advanced past its initial value.
Jun 19 02:39:36 dist-test-slave-zpbg krb5kdc[6610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836776, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.241.65@KRBTEST.COM
I20260619 02:39:36.799647 7130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:36.792456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41436 (local address 127.5.241.65:33231)
0619 02:39:36.792519 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:36.792522 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:36.792670 (+ 148us) server_negotiation.cc:407] Connection header received
0619 02:39:36.792701 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:36.792702 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:36.792712 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:36.792767 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:36.793326 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:36.793817 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:36.794538 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:36.794721 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:36.794861 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:36.797164 (+ 2303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:36.797165 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:36.797166 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:36.797173 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:36.798555 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:36.798986 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:36.798987 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:36.798987 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:36.799073 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:36.799240 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:36.799241 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:36.799241 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:36.799325 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:36.799396 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:36.799522 (+ 126us) server_negotiation.cc:299] Negotiation successful
0619 02:39:36.799584 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
Jun 19 02:39:36 dist-test-slave-zpbg krb5kdc[6610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836776, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 02:39:36 dist-test-slave-zpbg krb5kdc[6610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836776, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.241.65@KRBTEST.COM
I20260619 02:39:36.822482 7130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:36.815330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41440 (local address 127.5.241.65:33231)
0619 02:39:36.815409 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:36.815412 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:36.815485 (+ 73us) server_negotiation.cc:407] Connection header received
0619 02:39:36.815580 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:36.815581 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:36.815590 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:36.815647 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:36.816275 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:36.816763 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:36.817515 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:36.817718 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:36.817803 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:36.820096 (+ 2293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:36.820097 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:36.820097 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:36.820104 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:36.821391 (+ 1287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:36.821745 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:36.821746 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:36.821746 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:36.821805 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:36.822011 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:36.822012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:36.822012 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:36.822121 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:36.822194 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:36.822340 (+ 146us) server_negotiation.cc:299] Negotiation successful
0619 02:39:36.822412 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260619 02:39:36.823060 6760 tablet_service.cc:2908] Rejecting scan request for tablet 1ac7a2a46eb9444d904fcce518fa034c: Uninitialized: clean time has not yet been initialized
I20260619 02:39:36.823930 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 6717
I20260619 02:39:36.829473 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 6851
I20260619 02:39:36.834067 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 6985
I20260619 02:39:36.838384 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 6623
2026-06-19T02:39:36Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2042 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:39:36 dist-test-slave-zpbg krb5kdc[7155](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:39:36 dist-test-slave-zpbg krb5kdc[7155](info): set up 2 sockets
Jun 19 02:39:36 dist-test-slave-zpbg krb5kdc[7155](info): commencing operation
krb5kdc: starting...
W20260619 02:39:37.870432 6085 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.003s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Jun 19 02:39:37 dist-test-slave-zpbg krb5kdc[7155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836777, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:39:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:39:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.241.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
I20260619 02:39:37.989774 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:39901
--webserver_interface=127.5.241.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:33587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:39901
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:39:38.069510 7168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:39:38.069753 7168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:39:38.069787 7168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:39:38.070950 7168 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:39:38.071017 7168 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 02:39:38.071038 7168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:39:38.071061 7168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:39:38.071090 7168 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:39:38.071115 7168 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:39:38.072602 7168 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:33587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:39901
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:39901
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.241.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7168
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:39:38.072885 7168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:39:38.073115 7168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:39:38.078117 7173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:38.078171 7176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:38.078117 7174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:39:38.078395 7168 server_base.cc:1034] running on GCE node
I20260619 02:39:38.078663 7168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:39:38.079021 7168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:39:38.080181 7168 hybrid_clock.cc:648] HybridClock initialized: now 1781836778080162 us; error 32 us; skew 500 ppm
Jun 19 02:39:38 dist-test-slave-zpbg krb5kdc[7155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836778, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:39:38.087741 7168 init.cc:377] Logged in from keytab as kudu/127.5.241.126@KRBTEST.COM (short username kudu)
I20260619 02:39:38.088220 7168 webserver.cc:460] Webserver started at http://127.5.241.126:41441/ using document root <none> and password file <none>
I20260619 02:39:38.088459 7168 fs_manager.cc:359] Metadata directory not provided
I20260619 02:39:38.088505 7168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:39:38.088652 7168 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:39:38.089753 7168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/master-0/data/instance:
uuid: "87fa180e998d416da04956de2d1bd04c"
format_stamp: "Formatted at 2026-06-19 02:39:38 on dist-test-slave-zpbg"
server_key: "d212b955b98cca1f513f53896fc22752"
server_key_iv: "af79bbb4de816d9e0e60621192be038b"
server_key_version: "encryptionkey@0"
I20260619 02:39:38.090286 7168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/master-0/wal/instance:
uuid: "87fa180e998d416da04956de2d1bd04c"
format_stamp: "Formatted at 2026-06-19 02:39:38 on dist-test-slave-zpbg"
server_key: "d212b955b98cca1f513f53896fc22752"
server_key_iv: "af79bbb4de816d9e0e60621192be038b"
server_key_version: "encryptionkey@0"
I20260619 02:39:38.091802 7168 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:39:38.092628 7183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:38.093043 7168 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:39:38.093153 7168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "87fa180e998d416da04956de2d1bd04c"
format_stamp: "Formatted at 2026-06-19 02:39:38 on dist-test-slave-zpbg"
server_key: "d212b955b98cca1f513f53896fc22752"
server_key_iv: "af79bbb4de816d9e0e60621192be038b"
server_key_version: "encryptionkey@0"
I20260619 02:39:38.093243 7168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:39:38.108152 7168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:39:38.110553 7168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:39:38.110723 7168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:39:38.115288 7168 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.126:39901
I20260619 02:39:38.115399 7245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.126:39901 every 8 connection(s)
I20260619 02:39:38.115695 7168 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:39:38.116449 7246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:39:38.119230 7246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87fa180e998d416da04956de2d1bd04c: Bootstrap starting.
I20260619 02:39:38.119858 7246 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 87fa180e998d416da04956de2d1bd04c: Neither blocks nor log segments found. Creating new log.
I20260619 02:39:38.120183 7246 log.cc:824] T 00000000000000000000000000000000 P 87fa180e998d416da04956de2d1bd04c: Log is configured to *not* fsync() on all Append() calls
I20260619 02:39:38.120889 7246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87fa180e998d416da04956de2d1bd04c: No bootstrap required, opened a new log
I20260619 02:39:38.123128 7246 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87fa180e998d416da04956de2d1bd04c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87fa180e998d416da04956de2d1bd04c" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 39901 } }
I20260619 02:39:38.123274 7246 raft_consensus.cc:374] T 00000000000000000000000000000000 P 87fa180e998d416da04956de2d1bd04c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:39:38.123296 7246 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87fa180e998d416da04956de2d1bd04c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87fa180e998d416da04956de2d1bd04c, State: Initialized, Role: FOLLOWER
I20260619 02:39:38.123466 7246 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87fa180e998d416da04956de2d1bd04c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87fa180e998d416da04956de2d1bd04c" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 39901 } }
I20260619 02:39:38.123528 7246 raft_consensus.cc:388] T 00000000000000000000000000000000 P 87fa180e998d416da04956de2d1bd04c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:39:38.123566 7246 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87fa180e998d416da04956de2d1bd04c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:39:38.123616 7246 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87fa180e998d416da04956de2d1bd04c [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:39:38.124146 7246 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87fa180e998d416da04956de2d1bd04c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87fa180e998d416da04956de2d1bd04c" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 39901 } }
I20260619 02:39:38.124230 7246 leader_election.cc:302] T 00000000000000000000000000000000 P 87fa180e998d416da04956de2d1bd04c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 87fa180e998d416da04956de2d1bd04c; no voters:
I20260619 02:39:38.124414 7246 leader_election.cc:288] T 00000000000000000000000000000000 P 87fa180e998d416da04956de2d1bd04c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:39:38.124447 7249 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 87fa180e998d416da04956de2d1bd04c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:39:38.124440 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 7168
I20260619 02:39:38.124552 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/master-0/wal/instance
I20260619 02:39:38.124569 7249 raft_consensus.cc:686] T 00000000000000000000000000000000 P 87fa180e998d416da04956de2d1bd04c [term 1 LEADER]: Becoming Leader. State: Replica: 87fa180e998d416da04956de2d1bd04c, State: Running, Role: LEADER
I20260619 02:39:38.124681 7246 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87fa180e998d416da04956de2d1bd04c [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:39:38.124658 7249 consensus_queue.cc:237] T 00000000000000000000000000000000 P 87fa180e998d416da04956de2d1bd04c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87fa180e998d416da04956de2d1bd04c" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 39901 } }
I20260619 02:39:38.124744 6085 external_mini_cluster.cc:1428] Setting key f838937f93a6e0357b1579a345e80d78
I20260619 02:39:38.125041 7250 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87fa180e998d416da04956de2d1bd04c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87fa180e998d416da04956de2d1bd04c. Latest consensus state: current_term: 1 leader_uuid: "87fa180e998d416da04956de2d1bd04c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87fa180e998d416da04956de2d1bd04c" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 39901 } } }
I20260619 02:39:38.125131 7250 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87fa180e998d416da04956de2d1bd04c [sys.catalog]: This master's current role is: LEADER
I20260619 02:39:38.125401 7251 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87fa180e998d416da04956de2d1bd04c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "87fa180e998d416da04956de2d1bd04c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87fa180e998d416da04956de2d1bd04c" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 39901 } } }
I20260619 02:39:38.125517 7251 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87fa180e998d416da04956de2d1bd04c [sys.catalog]: This master's current role is: LEADER
I20260619 02:39:38.126060 7257 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:39:38.126730 7257 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:39:38.128387 7257 catalog_manager.cc:1347] Generated new cluster ID: c8cc613dfc9b4bf6a0242f669729830b
I20260619 02:39:38.128439 7257 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 02:39:38 dist-test-slave-zpbg krb5kdc[7155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836777, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:39:38.136375 7260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:38.125429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50818 (local address 127.5.241.126:39901)
0619 02:39:38.125663 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:38.125673 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:38.125906 (+ 233us) server_negotiation.cc:407] Connection header received
0619 02:39:38.126047 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:38.126056 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:38.126115 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:38.126280 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:38.126932 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.128138 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.129174 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.129811 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.129927 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:38.132788 (+ 2861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:38.132792 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:38.132792 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:38.132804 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:38.134722 (+ 1918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:38.135164 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:38.135167 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:38.135168 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:38.135268 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:38.135477 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:38.135478 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:38.135478 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:38.135761 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:38.135908 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:38.136035 (+ 127us) server_negotiation.cc:299] Negotiation successful
0619 02:39:38.136169 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":78,"threads_started":1}
I20260619 02:39:38.144645 7257 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:39:38.145257 7257 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:39:38.149250 7257 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 87fa180e998d416da04956de2d1bd04c: Generated new TSK 0
I20260619 02:39:38.149453 7257 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.241.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab.
Entry for principal kudu/127.5.241.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab.
I20260619 02:39:38.186127 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.65:0
--local_ip_for_outbound_sockets=127.5.241.65
--webserver_interface=127.5.241.65
--webserver_port=0
--tserver_master_addrs=127.5.241.126:39901
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab
--principal=kudu/127.5.241.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:33587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:39:38.288515 7272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:39:38.288750 7272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:39:38.288776 7272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:39:38.290042 7272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:39:38.290091 7272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:39:38.290112 7272 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 02:39:38.290133 7272 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.65
I20260619 02:39:38.291759 7272 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:33587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab
--principal=kudu/127.5.241.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.241.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.241.126:39901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7272
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.65
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:39:38.292017 7272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:39:38.292266 7272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:39:38.298439 7278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:38.298439 7280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:39:38.298730 7272 server_base.cc:1034] running on GCE node
W20260619 02:39:38.298976 7277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:39:38.299314 7272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:39:38.299604 7272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:39:38.300796 7272 hybrid_clock.cc:648] HybridClock initialized: now 1781836778300778 us; error 30 us; skew 500 ppm
Jun 19 02:39:38 dist-test-slave-zpbg krb5kdc[7155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836778, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:39:38.304633 7272 init.cc:377] Logged in from keytab as kudu/127.5.241.65@KRBTEST.COM (short username kudu)
I20260619 02:39:38.305096 7272 webserver.cc:460] Webserver started at http://127.5.241.65:43287/ using document root <none> and password file <none>
I20260619 02:39:38.305397 7272 fs_manager.cc:359] Metadata directory not provided
I20260619 02:39:38.305474 7272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:39:38.305615 7272 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:39:38.307080 7272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-0/data/instance:
uuid: "d8e295f235974861a5124ddc7d691d58"
format_stamp: "Formatted at 2026-06-19 02:39:38 on dist-test-slave-zpbg"
server_key: "9d594d83b328d411d1f180483c106b7a"
server_key_iv: "27132fe240a206b78dd490129f84ca18"
server_key_version: "encryptionkey@0"
I20260619 02:39:38.307511 7272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-0/wal/instance:
uuid: "d8e295f235974861a5124ddc7d691d58"
format_stamp: "Formatted at 2026-06-19 02:39:38 on dist-test-slave-zpbg"
server_key: "9d594d83b328d411d1f180483c106b7a"
server_key_iv: "27132fe240a206b78dd490129f84ca18"
server_key_version: "encryptionkey@0"
I20260619 02:39:38.308959 7272 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:39:38.309840 7287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:38.310111 7272 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:39:38.310231 7272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-0/wal
uuid: "d8e295f235974861a5124ddc7d691d58"
format_stamp: "Formatted at 2026-06-19 02:39:38 on dist-test-slave-zpbg"
server_key: "9d594d83b328d411d1f180483c106b7a"
server_key_iv: "27132fe240a206b78dd490129f84ca18"
server_key_version: "encryptionkey@0"
I20260619 02:39:38.310324 7272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:39:38.326745 7272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:39:38.329210 7272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:39:38.329377 7272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:39:38.329908 7272 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:39:38.329972 7272 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:38.330019 7272 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:39:38.330046 7272 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:38.337102 7272 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.65:46725
I20260619 02:39:38.337221 7402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.65:46725 every 8 connection(s)
Jun 19 02:39:38 dist-test-slave-zpbg krb5kdc[7155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836778, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.65@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:39:38.337520 7272 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
I20260619 02:39:38.340386 7260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:38.330453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:38755 (local address 127.5.241.126:39901)
0619 02:39:38.330528 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:38.330530 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:38.330747 (+ 217us) server_negotiation.cc:407] Connection header received
0619 02:39:38.331311 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:38.331313 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:38.331329 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:38.331414 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:38.332288 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.333144 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.334870 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.335204 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.335292 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:38.337854 (+ 2562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:38.337856 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:38.337856 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:38.337864 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:38.339178 (+ 1314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:38.339577 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:38.339578 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:38.339578 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:38.339627 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:38.339852 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:38.339852 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:38.339853 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:38.339949 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:38.339998 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:38.340261 (+ 263us) server_negotiation.cc:299] Negotiation successful
0619 02:39:38.340324 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 02:39:38.340631 7296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:38.330435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39901 (local address 127.5.241.65:38755)
0619 02:39:38.330612 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:38.330653 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:38.331205 (+ 552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:38.331487 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:38.331491 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:38.331863 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:39:38.332175 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:38.332184 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.333271 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.333272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:38.334705 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:38.334707 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.335294 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.335295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:38.335405 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:38.336019 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:39:38.336028 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:39:38.337741 (+ 1713us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:39:38.339273 (+ 1532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:38.339280 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:38.339281 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:38.339506 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:38.339694 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:38.339694 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:38.339695 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:38.339798 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:38.340000 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:39:38.340004 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:39:38.340148 (+ 144us) client_negotiation.cc:769] Sending connection context
0619 02:39:38.340240 (+ 92us) client_negotiation.cc:240] Negotiation successful
0619 02:39:38.340361 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":98,"threads_started":1}
I20260619 02:39:38.341187 7403 heartbeater.cc:347] Connected to a master server at 127.5.241.126:39901
I20260619 02:39:38.341266 7403 heartbeater.cc:464] Registering TS with master...
I20260619 02:39:38.341451 7403 heartbeater.cc:511] Master 127.5.241.126:39901 requested a full tablet report, sending...
I20260619 02:39:38.341984 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 7272
I20260619 02:39:38.342059 7200 ts_manager.cc:194] Registered new tserver with Master: d8e295f235974861a5124ddc7d691d58 (127.5.241.65:46725)
I20260619 02:39:38.342088 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-0/wal/instance
I20260619 02:39:38.342260 6085 external_mini_cluster.cc:1428] Setting key b77367a99902fe3bfbdbaa62163a4150
I20260619 02:39:38.342831 7200 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.65@KRBTEST.COM'} at 127.5.241.65:38755
I20260619 02:39:38.349128 7260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:38.342969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:35853 (local address 127.5.241.126:39901)
0619 02:39:38.343012 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:38.343014 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:38.343215 (+ 201us) server_negotiation.cc:407] Connection header received
0619 02:39:38.343288 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:38.343288 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:38.343296 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:38.343340 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:38.343740 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.344219 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.344954 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.345157 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.345243 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:38.346211 (+ 968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:38.346213 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:38.346213 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:38.346219 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:38.347811 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:38.348238 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:38.348239 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:38.348240 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:38.348307 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:38.348537 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:38.348538 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:38.348539 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:38.348673 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:38.348770 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:38.348945 (+ 175us) server_negotiation.cc:299] Negotiation successful
0619 02:39:38.349029 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 02:39:38.349145 7409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:38.343015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39901 (local address 127.5.241.65:35853)
0619 02:39:38.343154 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:38.343167 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:38.343229 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:38.343346 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:38.343346 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:38.343561 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:39:38.343630 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:38.343631 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.344336 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.344337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:38.344856 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:38.344857 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.345244 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.345245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:38.345316 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:38.345750 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:39:38.345754 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:39:38.346058 (+ 304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:39:38.347940 (+ 1882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:38.347941 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:38.347942 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:38.348143 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:38.348381 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:38.348383 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:38.348383 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:38.348450 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:38.348764 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:39:38.348765 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:39:38.348814 (+ 49us) client_negotiation.cc:769] Sending connection context
0619 02:39:38.348917 (+ 103us) client_negotiation.cc:240] Negotiation successful
0619 02:39:38.349029 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":67,"threads_started":1}
WARNING: no policy specified for kudu/127.5.241.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab.
Entry for principal kudu/127.5.241.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab.
I20260619 02:39:38.369798 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.66:0
--local_ip_for_outbound_sockets=127.5.241.66
--webserver_interface=127.5.241.66
--webserver_port=0
--tserver_master_addrs=127.5.241.126:39901
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab
--principal=kudu/127.5.241.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:33587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:39:38.450129 7412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:39:38.450366 7412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:39:38.450402 7412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:39:38.451637 7412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:39:38.451702 7412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:39:38.451736 7412 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 02:39:38.451776 7412 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.66
I20260619 02:39:38.453181 7412 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:33587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab
--principal=kudu/127.5.241.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.241.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.241.126:39901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7412
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.66
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:39:38.453442 7412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:39:38.453670 7412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:39:38.458530 7418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:38.458535 7420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:38.458595 7417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:39:38.458932 7412 server_base.cc:1034] running on GCE node
I20260619 02:39:38.459180 7412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:39:38.459431 7412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:39:38.460595 7412 hybrid_clock.cc:648] HybridClock initialized: now 1781836778460581 us; error 32 us; skew 500 ppm
Jun 19 02:39:38 dist-test-slave-zpbg krb5kdc[7155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836778, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:39:38.463282 7412 init.cc:377] Logged in from keytab as kudu/127.5.241.66@KRBTEST.COM (short username kudu)
I20260619 02:39:38.463780 7412 webserver.cc:460] Webserver started at http://127.5.241.66:33217/ using document root <none> and password file <none>
I20260619 02:39:38.464048 7412 fs_manager.cc:359] Metadata directory not provided
I20260619 02:39:38.464120 7412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:39:38.464246 7412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:39:38.465591 7412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-1/data/instance:
uuid: "8d02cf2496944f959f15c078c2f7ff5a"
format_stamp: "Formatted at 2026-06-19 02:39:38 on dist-test-slave-zpbg"
server_key: "5f0b02279c4cb4847e534f53c59f820d"
server_key_iv: "0bd10101e96c6731cfa4a091fd902a3b"
server_key_version: "encryptionkey@0"
I20260619 02:39:38.465893 7412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-1/wal/instance:
uuid: "8d02cf2496944f959f15c078c2f7ff5a"
format_stamp: "Formatted at 2026-06-19 02:39:38 on dist-test-slave-zpbg"
server_key: "5f0b02279c4cb4847e534f53c59f820d"
server_key_iv: "0bd10101e96c6731cfa4a091fd902a3b"
server_key_version: "encryptionkey@0"
I20260619 02:39:38.467216 7412 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:39:38.467905 7427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:38.468081 7412 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:39:38.468176 7412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-1/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-1/wal
uuid: "8d02cf2496944f959f15c078c2f7ff5a"
format_stamp: "Formatted at 2026-06-19 02:39:38 on dist-test-slave-zpbg"
server_key: "5f0b02279c4cb4847e534f53c59f820d"
server_key_iv: "0bd10101e96c6731cfa4a091fd902a3b"
server_key_version: "encryptionkey@0"
I20260619 02:39:38.468262 7412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:39:38.482754 7412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:39:38.485237 7412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:39:38.485416 7412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:39:38.485946 7412 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:39:38.486018 7412 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:38.486073 7412 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:39:38.486104 7412 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:38.492882 7412 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.66:33659
I20260619 02:39:38.492970 7542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.66:33659 every 8 connection(s)
Jun 19 02:39:38 dist-test-slave-zpbg krb5kdc[7155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836778, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.66@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:39:38.493324 7412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
I20260619 02:39:38.494966 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 7412
I20260619 02:39:38.495062 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-1/wal/instance
I20260619 02:39:38.495245 6085 external_mini_cluster.cc:1428] Setting key 7521280db6669eae54796579efb5a827
I20260619 02:39:38.496990 7260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:38.486440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:56403 (local address 127.5.241.126:39901)
0619 02:39:38.486483 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:38.486487 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:38.486703 (+ 216us) server_negotiation.cc:407] Connection header received
0619 02:39:38.487308 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:38.487309 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:38.487322 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:38.487381 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:38.488089 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.488898 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.490586 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.490855 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.490950 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:38.493734 (+ 2784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:38.493736 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:38.493736 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:38.493746 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:38.495418 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:38.495944 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:38.495945 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:38.495950 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:38.496007 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:38.496284 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:38.496284 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:38.496284 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:38.496380 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:38.496425 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:38.496806 (+ 381us) server_negotiation.cc:299] Negotiation successful
0619 02:39:38.496906 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 02:39:38.497223 7436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:38.486438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39901 (local address 127.5.241.66:56403)
0619 02:39:38.486575 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:38.486611 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:38.487192 (+ 581us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:38.487457 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:38.487460 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:38.487799 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:39:38.487987 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:38.487992 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.488990 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.488991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:38.490503 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:38.490506 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.490957 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.490958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:38.491106 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:38.491686 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:39:38.491698 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:39:38.493620 (+ 1922us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:39:38.495507 (+ 1887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:38.495514 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:38.495516 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:38.495861 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:38.496073 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:38.496075 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:38.496075 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:38.496222 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:38.496462 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:39:38.496467 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:39:38.496636 (+ 169us) client_negotiation.cc:769] Sending connection context
0619 02:39:38.496772 (+ 136us) client_negotiation.cc:240] Negotiation successful
0619 02:39:38.496948 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":77,"threads_started":1}
I20260619 02:39:38.497714 7543 heartbeater.cc:347] Connected to a master server at 127.5.241.126:39901
I20260619 02:39:38.497799 7543 heartbeater.cc:464] Registering TS with master...
I20260619 02:39:38.497990 7543 heartbeater.cc:511] Master 127.5.241.126:39901 requested a full tablet report, sending...
I20260619 02:39:38.498330 7199 ts_manager.cc:194] Registered new tserver with Master: 8d02cf2496944f959f15c078c2f7ff5a (127.5.241.66:33659)
I20260619 02:39:38.498828 7199 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.66@KRBTEST.COM'} at 127.5.241.66:56403
WARNING: no policy specified for kudu/127.5.241.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.67@KRBTEST.COM" created.
I20260619 02:39:38.508773 7550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:38.498745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39901 (local address 127.5.241.66:59619)
0619 02:39:38.502890 (+ 4145us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:38.502907 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:38.502986 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:38.503118 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:38.503119 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:38.503298 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:39:38.503350 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:38.503351 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.504013 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.504015 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:38.504584 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:38.504586 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.504967 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.504968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:38.505047 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:38.505544 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:39:38.505549 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:39:38.505977 (+ 428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:39:38.507665 (+ 1688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:38.507666 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:38.507667 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:38.507840 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:38.508052 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:38.508053 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:38.508053 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:38.508111 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:38.508413 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:39:38.508414 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:39:38.508466 (+ 52us) client_negotiation.cc:769] Sending connection context
0619 02:39:38.508566 (+ 100us) client_negotiation.cc:240] Negotiation successful
0619 02:39:38.508678 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4111,"thread_start_us":97,"threads_started":1}
I20260619 02:39:38.508769 7260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:38.498835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:59619 (local address 127.5.241.126:39901)
0619 02:39:38.498884 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:38.498886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:38.502973 (+ 4087us) server_negotiation.cc:407] Connection header received
0619 02:39:38.503047 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:38.503048 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:38.503058 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:38.503121 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:38.503427 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.503921 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.504693 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.504882 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.504966 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:38.506090 (+ 1124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:38.506092 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:38.506092 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:38.506098 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:38.507569 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:38.507927 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:38.507928 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:38.507929 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:38.507985 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:38.508184 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:38.508185 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:38.508185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:38.508335 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:38.508411 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:38.508595 (+ 184us) server_negotiation.cc:299] Negotiation successful
0619 02:39:38.508672 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab.
Entry for principal kudu/127.5.241.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab.
I20260619 02:39:38.521485 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.67:0
--local_ip_for_outbound_sockets=127.5.241.67
--webserver_interface=127.5.241.67
--webserver_port=0
--tserver_master_addrs=127.5.241.126:39901
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab
--principal=kudu/127.5.241.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:33587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:39:38.601454 7552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:39:38.601693 7552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:39:38.601732 7552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:39:38.602958 7552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:39:38.603017 7552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:39:38.603055 7552 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 02:39:38.603091 7552 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.67
I20260619 02:39:38.604539 7552 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:33587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab
--principal=kudu/127.5.241.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.241.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.241.126:39901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7552
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.67
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:39:38.604799 7552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:39:38.605019 7552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:39:38.609862 7558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:39:38.609969 7552 server_base.cc:1034] running on GCE node
W20260619 02:39:38.609862 7557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:38.609925 7560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:39:38.610322 7552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:39:38.610569 7552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:39:38.611730 7552 hybrid_clock.cc:648] HybridClock initialized: now 1781836778611727 us; error 36 us; skew 500 ppm
Jun 19 02:39:38 dist-test-slave-zpbg krb5kdc[7155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836778, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:39:38.614338 7552 init.cc:377] Logged in from keytab as kudu/127.5.241.67@KRBTEST.COM (short username kudu)
I20260619 02:39:38.614825 7552 webserver.cc:460] Webserver started at http://127.5.241.67:34977/ using document root <none> and password file <none>
I20260619 02:39:38.615056 7552 fs_manager.cc:359] Metadata directory not provided
I20260619 02:39:38.615123 7552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:39:38.615258 7552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:39:38.616365 7552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-2/data/instance:
uuid: "00218ac386364021825bd67e6c27101e"
format_stamp: "Formatted at 2026-06-19 02:39:38 on dist-test-slave-zpbg"
server_key: "712853b54fdbfd7a64bdee30587477f7"
server_key_iv: "3689cc06e2d02a5ba55a549d8a953db5"
server_key_version: "encryptionkey@0"
I20260619 02:39:38.616724 7552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-2/wal/instance:
uuid: "00218ac386364021825bd67e6c27101e"
format_stamp: "Formatted at 2026-06-19 02:39:38 on dist-test-slave-zpbg"
server_key: "712853b54fdbfd7a64bdee30587477f7"
server_key_iv: "3689cc06e2d02a5ba55a549d8a953db5"
server_key_version: "encryptionkey@0"
I20260619 02:39:38.618019 7552 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:39:38.618878 7567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:38.619058 7552 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:39:38.619153 7552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-2/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-2/wal
uuid: "00218ac386364021825bd67e6c27101e"
format_stamp: "Formatted at 2026-06-19 02:39:38 on dist-test-slave-zpbg"
server_key: "712853b54fdbfd7a64bdee30587477f7"
server_key_iv: "3689cc06e2d02a5ba55a549d8a953db5"
server_key_version: "encryptionkey@0"
I20260619 02:39:38.619251 7552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:39:38.635798 7552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:39:38.638150 7552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:39:38.638327 7552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:39:38.638911 7552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:39:38.638963 7552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:38.639007 7552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:39:38.639039 7552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:38.645972 7552 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.67:41185
I20260619 02:39:38.646044 7682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.67:41185 every 8 connection(s)
I20260619 02:39:38.646335 7552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
Jun 19 02:39:38 dist-test-slave-zpbg krb5kdc[7155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836778, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.67@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:39:38.646623 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 7552
I20260619 02:39:38.646708 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-2/wal/instance
I20260619 02:39:38.646934 6085 external_mini_cluster.cc:1428] Setting key 5b02799f65f1d7504e97c41a725e5ddd
I20260619 02:39:38.649938 7260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:38.639482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:43343 (local address 127.5.241.126:39901)
0619 02:39:38.639552 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:38.639555 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:38.639786 (+ 231us) server_negotiation.cc:407] Connection header received
0619 02:39:38.640573 (+ 787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:38.640575 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:38.640588 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:38.640660 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:38.641582 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.642412 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.644152 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.644337 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.644394 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:38.647064 (+ 2670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:38.647066 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:38.647066 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:38.647074 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:38.648537 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:38.648962 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:38.648963 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:38.648964 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:38.649015 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:38.649328 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:38.649330 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:38.649330 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:38.649462 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:38.649518 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:38.649799 (+ 281us) server_negotiation.cc:299] Negotiation successful
0619 02:39:38.649877 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 02:39:38.650046 7576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:38.639462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39901 (local address 127.5.241.67:43343)
0619 02:39:38.639638 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:38.639687 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:38.640434 (+ 747us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:38.640763 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:38.640767 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:38.641117 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:39:38.641454 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:38.641461 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.642518 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.642520 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:38.644064 (+ 1544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:38.644067 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.644424 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.644425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:38.644581 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:38.645169 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:39:38.645181 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:39:38.646953 (+ 1772us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:39:38.648623 (+ 1670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:38.648629 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:38.648631 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:38.648878 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:38.649076 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:38.649077 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:38.649078 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:38.649225 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:38.649530 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:39:38.649535 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:39:38.649664 (+ 129us) client_negotiation.cc:769] Sending connection context
0619 02:39:38.649750 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 02:39:38.649864 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":89,"threads_started":1}
I20260619 02:39:38.650548 7683 heartbeater.cc:347] Connected to a master server at 127.5.241.126:39901
I20260619 02:39:38.650640 7683 heartbeater.cc:464] Registering TS with master...
I20260619 02:39:38.650836 7683 heartbeater.cc:511] Master 127.5.241.126:39901 requested a full tablet report, sending...
I20260619 02:39:38.651262 7200 ts_manager.cc:194] Registered new tserver with Master: 00218ac386364021825bd67e6c27101e (127.5.241.67:41185)
I20260619 02:39:38.651671 7200 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.67@KRBTEST.COM'} at 127.5.241.67:43343
I20260619 02:39:38.656591 7260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:38.651691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:53187 (local address 127.5.241.126:39901)
0619 02:39:38.651734 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:38.651743 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:38.651800 (+ 57us) server_negotiation.cc:407] Connection header received
0619 02:39:38.651865 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:38.651866 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:38.651876 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:38.651931 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:38.652299 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.652731 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.653433 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.653606 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.653656 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:38.654520 (+ 864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:38.654521 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:38.654521 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:38.654526 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:38.655757 (+ 1231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:38.656038 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:38.656039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:38.656039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:38.656082 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:38.656261 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:38.656262 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:38.656262 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:38.656343 (+ 81us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:38.656386 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:38.656483 (+ 97us) server_negotiation.cc:299] Negotiation successful
0619 02:39:38.656534 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 02:39:38.656607 7689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:38.651634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39901 (local address 127.5.241.67:53187)
0619 02:39:38.651744 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:38.651755 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:38.651812 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:38.651934 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:38.651934 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:38.652160 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:39:38.652217 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:38.652218 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.652831 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.652832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:38.653362 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:38.653363 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.653664 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.653665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:38.653716 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:38.654143 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:39:38.654148 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:39:38.654448 (+ 300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:39:38.655821 (+ 1373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:38.655822 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:38.655822 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:38.655978 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:38.656133 (+ 155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:38.656133 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:38.656133 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:38.656200 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:38.656387 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:39:38.656387 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:39:38.656420 (+ 33us) client_negotiation.cc:769] Sending connection context
0619 02:39:38.656469 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 02:39:38.656540 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":63,"threads_started":1}
I20260619 02:39:38.658603 6085 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 02:39:38 dist-test-slave-zpbg krb5kdc[7155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836778, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 02:39:38 dist-test-slave-zpbg krb5kdc[7155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836778, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:39:38.681591 7260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:38.674715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50832 (local address 127.5.241.126:39901)
0619 02:39:38.674819 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:38.674821 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:38.674848 (+ 27us) server_negotiation.cc:407] Connection header received
0619 02:39:38.674878 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:38.674879 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:38.674896 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:38.674940 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:38.675488 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.675971 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.676651 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.676829 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.676880 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:38.679105 (+ 2225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:38.679107 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:38.679107 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:38.679114 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:38.680572 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:38.680908 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:38.680909 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:38.680909 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:38.680957 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:38.681204 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:38.681204 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:38.681205 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:38.681303 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:38.681380 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:38.681468 (+ 88us) server_negotiation.cc:299] Negotiation successful
0619 02:39:38.681525 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 02:39:38.683465 7200 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50832:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:39:38.685026 7200 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:39:38.691737 7703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:38.687587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55084 (local address 127.5.241.65:46725)
0619 02:39:38.687746 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:38.687751 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:38.687997 (+ 246us) server_negotiation.cc:407] Connection header received
0619 02:39:38.688066 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:38.688068 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:38.688094 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:38.688318 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:39:38.688370 (+ 52us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.689145 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.690832 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.691319 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.691416 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:38.691488 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:38.691598 (+ 110us) server_negotiation.cc:299] Negotiation successful
0619 02:39:38.691637 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":101,"threads_started":1}
I20260619 02:39:38.691725 7704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:38.687448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:46725 (local address 127.0.0.1:55084)
0619 02:39:38.687933 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:38.687942 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:38.688000 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:38.688194 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:38.688195 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:38.688197 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:39:38.688244 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:38.688246 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.689226 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.689227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:38.690619 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:38.690621 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.691418 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.691419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:38.691498 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:38.691511 (+ 13us) client_negotiation.cc:769] Sending connection context
0619 02:39:38.691584 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 02:39:38.691640 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"mutex_wait_us":56,"thread_start_us":83,"threads_started":1}
I20260619 02:39:38.691895 7705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:38.687766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33182 (local address 127.5.241.67:41185)
0619 02:39:38.687914 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:38.687919 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:38.688208 (+ 289us) server_negotiation.cc:407] Connection header received
0619 02:39:38.688248 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:38.688250 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:38.688266 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:38.688331 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:39:38.688522 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.689507 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.690605 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.691380 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.691452 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:38.691504 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:38.691760 (+ 256us) server_negotiation.cc:299] Negotiation successful
0619 02:39:38.691800 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":87,"threads_started":1}
I20260619 02:39:38.691907 7701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:38.687371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53756 (local address 127.5.241.66:33659)
0619 02:39:38.687794 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:38.687797 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:38.687806 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:39:38.687848 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:38.687849 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:38.687862 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:38.687958 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:39:38.688114 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.689244 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.690681 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.691385 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.691452 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:38.691504 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:38.691778 (+ 274us) server_negotiation.cc:299] Negotiation successful
0619 02:39:38.691814 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":107,"threads_started":1}
I20260619 02:39:38.691949 7700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:38.687255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.66:33659 (local address 127.0.0.1:53756)
0619 02:39:38.687356 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:38.687397 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:38.687469 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:38.687961 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:38.687964 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:38.687967 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:39:38.688031 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:38.688033 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.689311 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.689312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:38.690603 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:38.690606 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.691541 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.691543 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:38.691624 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:38.691627 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:39:38.691671 (+ 44us) client_negotiation.cc:240] Negotiation successful
0619 02:39:38.691716 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77,"mutex_wait_us":76,"thread_start_us":57,"threads_started":1}
I20260619 02:39:38.692106 7702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:38.687643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.67:41185 (local address 127.0.0.1:33182)
0619 02:39:38.688097 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:38.688106 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:38.688149 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:38.688393 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:38.688394 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:38.688397 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:39:38.688434 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:38.688435 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.689588 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.689589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:38.690512 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:38.690515 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.691543 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.691544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:38.691615 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:38.691620 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 02:39:38.691671 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 02:39:38.691718 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"mutex_wait_us":5,"thread_start_us":65,"threads_started":1}
I20260619 02:39:38.692734 7617 tablet_service.cc:1467] Processing CreateTablet for tablet 1a33c6f14953460aaea887317b805e2d (DEFAULT_TABLE table=test-table [id=b3be3dbaff5147c8a99186e2decd85fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:39:38.692734 7477 tablet_service.cc:1467] Processing CreateTablet for tablet 1a33c6f14953460aaea887317b805e2d (DEFAULT_TABLE table=test-table [id=b3be3dbaff5147c8a99186e2decd85fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:39:38.693084 7617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a33c6f14953460aaea887317b805e2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:39:38.693089 7477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a33c6f14953460aaea887317b805e2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:39:38.693027 7337 tablet_service.cc:1467] Processing CreateTablet for tablet 1a33c6f14953460aaea887317b805e2d (DEFAULT_TABLE table=test-table [id=b3be3dbaff5147c8a99186e2decd85fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:39:38.693303 7337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a33c6f14953460aaea887317b805e2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:39:38.695725 7706 tablet_bootstrap.cc:492] T 1a33c6f14953460aaea887317b805e2d P 00218ac386364021825bd67e6c27101e: Bootstrap starting.
I20260619 02:39:38.696722 7706 tablet_bootstrap.cc:654] T 1a33c6f14953460aaea887317b805e2d P 00218ac386364021825bd67e6c27101e: Neither blocks nor log segments found. Creating new log.
I20260619 02:39:38.696707 7707 tablet_bootstrap.cc:492] T 1a33c6f14953460aaea887317b805e2d P d8e295f235974861a5124ddc7d691d58: Bootstrap starting.
I20260619 02:39:38.697098 7706 log.cc:824] T 1a33c6f14953460aaea887317b805e2d P 00218ac386364021825bd67e6c27101e: Log is configured to *not* fsync() on all Append() calls
I20260619 02:39:38.697357 7707 tablet_bootstrap.cc:654] T 1a33c6f14953460aaea887317b805e2d P d8e295f235974861a5124ddc7d691d58: Neither blocks nor log segments found. Creating new log.
I20260619 02:39:38.697644 7707 log.cc:824] T 1a33c6f14953460aaea887317b805e2d P d8e295f235974861a5124ddc7d691d58: Log is configured to *not* fsync() on all Append() calls
I20260619 02:39:38.697885 7706 tablet_bootstrap.cc:492] T 1a33c6f14953460aaea887317b805e2d P 00218ac386364021825bd67e6c27101e: No bootstrap required, opened a new log
I20260619 02:39:38.697956 7706 ts_tablet_manager.cc:1397] T 1a33c6f14953460aaea887317b805e2d P 00218ac386364021825bd67e6c27101e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:39:38.697983 7708 tablet_bootstrap.cc:492] T 1a33c6f14953460aaea887317b805e2d P 8d02cf2496944f959f15c078c2f7ff5a: Bootstrap starting.
I20260619 02:39:38.698418 7707 tablet_bootstrap.cc:492] T 1a33c6f14953460aaea887317b805e2d P d8e295f235974861a5124ddc7d691d58: No bootstrap required, opened a new log
I20260619 02:39:38.698484 7707 ts_tablet_manager.cc:1397] T 1a33c6f14953460aaea887317b805e2d P d8e295f235974861a5124ddc7d691d58: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:39:38.698622 7708 tablet_bootstrap.cc:654] T 1a33c6f14953460aaea887317b805e2d P 8d02cf2496944f959f15c078c2f7ff5a: Neither blocks nor log segments found. Creating new log.
I20260619 02:39:38.699024 7708 log.cc:824] T 1a33c6f14953460aaea887317b805e2d P 8d02cf2496944f959f15c078c2f7ff5a: Log is configured to *not* fsync() on all Append() calls
I20260619 02:39:38.699787 7708 tablet_bootstrap.cc:492] T 1a33c6f14953460aaea887317b805e2d P 8d02cf2496944f959f15c078c2f7ff5a: No bootstrap required, opened a new log
I20260619 02:39:38.699854 7708 ts_tablet_manager.cc:1397] T 1a33c6f14953460aaea887317b805e2d P 8d02cf2496944f959f15c078c2f7ff5a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:39:38.700019 7706 raft_consensus.cc:348] T 1a33c6f14953460aaea887317b805e2d P 00218ac386364021825bd67e6c27101e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d02cf2496944f959f15c078c2f7ff5a" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 33659 } } peers { permanent_uuid: "d8e295f235974861a5124ddc7d691d58" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46725 } } peers { permanent_uuid: "00218ac386364021825bd67e6c27101e" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 41185 } }
I20260619 02:39:38.700193 7706 raft_consensus.cc:374] T 1a33c6f14953460aaea887317b805e2d P 00218ac386364021825bd67e6c27101e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:39:38.700234 7706 raft_consensus.cc:729] T 1a33c6f14953460aaea887317b805e2d P 00218ac386364021825bd67e6c27101e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00218ac386364021825bd67e6c27101e, State: Initialized, Role: FOLLOWER
I20260619 02:39:38.700345 7706 consensus_queue.cc:260] T 1a33c6f14953460aaea887317b805e2d P 00218ac386364021825bd67e6c27101e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d02cf2496944f959f15c078c2f7ff5a" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 33659 } } peers { permanent_uuid: "d8e295f235974861a5124ddc7d691d58" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46725 } } peers { permanent_uuid: "00218ac386364021825bd67e6c27101e" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 41185 } }
I20260619 02:39:38.700320 7707 raft_consensus.cc:348] T 1a33c6f14953460aaea887317b805e2d P d8e295f235974861a5124ddc7d691d58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d02cf2496944f959f15c078c2f7ff5a" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 33659 } } peers { permanent_uuid: "d8e295f235974861a5124ddc7d691d58" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46725 } } peers { permanent_uuid: "00218ac386364021825bd67e6c27101e" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 41185 } }
I20260619 02:39:38.700493 7707 raft_consensus.cc:374] T 1a33c6f14953460aaea887317b805e2d P d8e295f235974861a5124ddc7d691d58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:39:38.700526 7707 raft_consensus.cc:729] T 1a33c6f14953460aaea887317b805e2d P d8e295f235974861a5124ddc7d691d58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8e295f235974861a5124ddc7d691d58, State: Initialized, Role: FOLLOWER
I20260619 02:39:38.700591 7706 ts_tablet_manager.cc:1428] T 1a33c6f14953460aaea887317b805e2d P 00218ac386364021825bd67e6c27101e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 02:39:38.700644 7707 consensus_queue.cc:260] T 1a33c6f14953460aaea887317b805e2d P d8e295f235974861a5124ddc7d691d58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d02cf2496944f959f15c078c2f7ff5a" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 33659 } } peers { permanent_uuid: "d8e295f235974861a5124ddc7d691d58" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46725 } } peers { permanent_uuid: "00218ac386364021825bd67e6c27101e" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 41185 } }
I20260619 02:39:38.700704 7683 heartbeater.cc:503] Master 127.5.241.126:39901 was elected leader, sending a full tablet report...
I20260619 02:39:38.700943 7403 heartbeater.cc:503] Master 127.5.241.126:39901 was elected leader, sending a full tablet report...
I20260619 02:39:38.701215 7707 ts_tablet_manager.cc:1428] T 1a33c6f14953460aaea887317b805e2d P d8e295f235974861a5124ddc7d691d58: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 02:39:38.701671 7708 raft_consensus.cc:348] T 1a33c6f14953460aaea887317b805e2d P 8d02cf2496944f959f15c078c2f7ff5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d02cf2496944f959f15c078c2f7ff5a" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 33659 } } peers { permanent_uuid: "d8e295f235974861a5124ddc7d691d58" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46725 } } peers { permanent_uuid: "00218ac386364021825bd67e6c27101e" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 41185 } }
I20260619 02:39:38.701853 7708 raft_consensus.cc:374] T 1a33c6f14953460aaea887317b805e2d P 8d02cf2496944f959f15c078c2f7ff5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:39:38.701907 7708 raft_consensus.cc:729] T 1a33c6f14953460aaea887317b805e2d P 8d02cf2496944f959f15c078c2f7ff5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d02cf2496944f959f15c078c2f7ff5a, State: Initialized, Role: FOLLOWER
I20260619 02:39:38.702016 7708 consensus_queue.cc:260] T 1a33c6f14953460aaea887317b805e2d P 8d02cf2496944f959f15c078c2f7ff5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d02cf2496944f959f15c078c2f7ff5a" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 33659 } } peers { permanent_uuid: "d8e295f235974861a5124ddc7d691d58" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46725 } } peers { permanent_uuid: "00218ac386364021825bd67e6c27101e" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 41185 } }
I20260619 02:39:38.702266 7708 ts_tablet_manager.cc:1428] T 1a33c6f14953460aaea887317b805e2d P 8d02cf2496944f959f15c078c2f7ff5a: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 02:39:38.702333 7543 heartbeater.cc:503] Master 127.5.241.126:39901 was elected leader, sending a full tablet report...
I20260619 02:39:38.704275 7714 raft_consensus.cc:482] T 1a33c6f14953460aaea887317b805e2d P 8d02cf2496944f959f15c078c2f7ff5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:39:38.704355 7714 raft_consensus.cc:504] T 1a33c6f14953460aaea887317b805e2d P 8d02cf2496944f959f15c078c2f7ff5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d02cf2496944f959f15c078c2f7ff5a" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 33659 } } peers { permanent_uuid: "d8e295f235974861a5124ddc7d691d58" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46725 } } peers { permanent_uuid: "00218ac386364021825bd67e6c27101e" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 41185 } }
I20260619 02:39:38.704596 7714 leader_election.cc:288] T 1a33c6f14953460aaea887317b805e2d P 8d02cf2496944f959f15c078c2f7ff5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8e295f235974861a5124ddc7d691d58 (127.5.241.65:46725), 00218ac386364021825bd67e6c27101e (127.5.241.67:41185)
I20260619 02:39:38.707566 7705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:38.704952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:45977 (local address 127.5.241.67:41185)
0619 02:39:38.704992 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:38.704994 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:38.705036 (+ 42us) server_negotiation.cc:407] Connection header received
0619 02:39:38.705103 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:38.705110 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:38.705118 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:38.705169 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:39:38.705312 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.705773 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.706863 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.707216 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.707305 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:38.707314 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:38.707466 (+ 152us) server_negotiation.cc:299] Negotiation successful
0619 02:39:38.707502 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 02:39:38.707594 7703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:38.704801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:36393 (local address 127.5.241.65:46725)
0619 02:39:38.704844 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:38.704846 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:38.704856 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:39:38.704922 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:38.704923 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:38.704932 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:38.704987 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:39:38.705168 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.705695 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.706867 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.707231 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.707323 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:38.707330 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:38.707499 (+ 169us) server_negotiation.cc:299] Negotiation successful
0619 02:39:38.707534 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 02:39:38.707614 7436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:38.704739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:46725 (local address 127.5.241.66:36393)
0619 02:39:38.704792 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:38.704804 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:38.704865 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:38.705039 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:38.705040 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:38.705042 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:39:38.705089 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:38.705091 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.705778 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.705779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:38.706737 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:38.706740 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.707323 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.707323 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:38.707431 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:38.707435 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:39:38.707492 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 02:39:38.707530 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":3}
I20260619 02:39:38.707782 7715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:38.704866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.67:41185 (local address 127.5.241.66:45977)
0619 02:39:38.704987 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:38.704998 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:38.705049 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:38.705172 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:38.705173 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:38.705174 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:39:38.705247 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:38.705249 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.705854 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.705855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:38.706746 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:38.706749 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.707303 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.707304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:38.707387 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:38.707390 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:39:38.707459 (+ 69us) client_negotiation.cc:240] Negotiation successful
0619 02:39:38.707496 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"mutex_wait_us":3,"thread_start_us":79,"threads_started":1}
I20260619 02:39:38.707944 7357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a33c6f14953460aaea887317b805e2d" candidate_uuid: "8d02cf2496944f959f15c078c2f7ff5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8e295f235974861a5124ddc7d691d58" is_pre_election: true
I20260619 02:39:38.708071 7357 raft_consensus.cc:2454] T 1a33c6f14953460aaea887317b805e2d P d8e295f235974861a5124ddc7d691d58 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d02cf2496944f959f15c078c2f7ff5a in term 0.
I20260619 02:39:38.708062 7637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a33c6f14953460aaea887317b805e2d" candidate_uuid: "8d02cf2496944f959f15c078c2f7ff5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00218ac386364021825bd67e6c27101e" is_pre_election: true
I20260619 02:39:38.708189 7637 raft_consensus.cc:2454] T 1a33c6f14953460aaea887317b805e2d P 00218ac386364021825bd67e6c27101e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d02cf2496944f959f15c078c2f7ff5a in term 0.
I20260619 02:39:38.708292 7431 leader_election.cc:302] T 1a33c6f14953460aaea887317b805e2d P 8d02cf2496944f959f15c078c2f7ff5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d02cf2496944f959f15c078c2f7ff5a, d8e295f235974861a5124ddc7d691d58; no voters:
I20260619 02:39:38.708436 7714 raft_consensus.cc:2781] T 1a33c6f14953460aaea887317b805e2d P 8d02cf2496944f959f15c078c2f7ff5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:39:38.708513 7714 raft_consensus.cc:482] T 1a33c6f14953460aaea887317b805e2d P 8d02cf2496944f959f15c078c2f7ff5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:39:38.708537 7714 raft_consensus.cc:3037] T 1a33c6f14953460aaea887317b805e2d P 8d02cf2496944f959f15c078c2f7ff5a [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:39:38.709364 7714 raft_consensus.cc:504] T 1a33c6f14953460aaea887317b805e2d P 8d02cf2496944f959f15c078c2f7ff5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d02cf2496944f959f15c078c2f7ff5a" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 33659 } } peers { permanent_uuid: "d8e295f235974861a5124ddc7d691d58" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46725 } } peers { permanent_uuid: "00218ac386364021825bd67e6c27101e" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 41185 } }
I20260619 02:39:38.709487 7714 leader_election.cc:288] T 1a33c6f14953460aaea887317b805e2d P 8d02cf2496944f959f15c078c2f7ff5a [CANDIDATE]: Term 1 election: Requested vote from peers d8e295f235974861a5124ddc7d691d58 (127.5.241.65:46725), 00218ac386364021825bd67e6c27101e (127.5.241.67:41185)
I20260619 02:39:38.709659 7637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a33c6f14953460aaea887317b805e2d" candidate_uuid: "8d02cf2496944f959f15c078c2f7ff5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00218ac386364021825bd67e6c27101e"
I20260619 02:39:38.709646 7357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a33c6f14953460aaea887317b805e2d" candidate_uuid: "8d02cf2496944f959f15c078c2f7ff5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8e295f235974861a5124ddc7d691d58"
I20260619 02:39:38.709729 7357 raft_consensus.cc:3037] T 1a33c6f14953460aaea887317b805e2d P d8e295f235974861a5124ddc7d691d58 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:39:38.709729 7637 raft_consensus.cc:3037] T 1a33c6f14953460aaea887317b805e2d P 00218ac386364021825bd67e6c27101e [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:39:38.710523 7637 raft_consensus.cc:2454] T 1a33c6f14953460aaea887317b805e2d P 00218ac386364021825bd67e6c27101e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d02cf2496944f959f15c078c2f7ff5a in term 1.
I20260619 02:39:38.710522 7357 raft_consensus.cc:2454] T 1a33c6f14953460aaea887317b805e2d P d8e295f235974861a5124ddc7d691d58 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d02cf2496944f959f15c078c2f7ff5a in term 1.
I20260619 02:39:38.710747 7431 leader_election.cc:302] T 1a33c6f14953460aaea887317b805e2d P 8d02cf2496944f959f15c078c2f7ff5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00218ac386364021825bd67e6c27101e, 8d02cf2496944f959f15c078c2f7ff5a; no voters:
I20260619 02:39:38.710845 7714 raft_consensus.cc:2781] T 1a33c6f14953460aaea887317b805e2d P 8d02cf2496944f959f15c078c2f7ff5a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:39:38.710973 7714 raft_consensus.cc:686] T 1a33c6f14953460aaea887317b805e2d P 8d02cf2496944f959f15c078c2f7ff5a [term 1 LEADER]: Becoming Leader. State: Replica: 8d02cf2496944f959f15c078c2f7ff5a, State: Running, Role: LEADER
I20260619 02:39:38.711059 7714 consensus_queue.cc:237] T 1a33c6f14953460aaea887317b805e2d P 8d02cf2496944f959f15c078c2f7ff5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d02cf2496944f959f15c078c2f7ff5a" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 33659 } } peers { permanent_uuid: "d8e295f235974861a5124ddc7d691d58" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46725 } } peers { permanent_uuid: "00218ac386364021825bd67e6c27101e" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 41185 } }
I20260619 02:39:38.711853 7199 catalog_manager.cc:5495] T 1a33c6f14953460aaea887317b805e2d P 8d02cf2496944f959f15c078c2f7ff5a reported cstate change: term changed from 0 to 1, leader changed from <none> to 8d02cf2496944f959f15c078c2f7ff5a (127.5.241.66). New cstate: current_term: 1 leader_uuid: "8d02cf2496944f959f15c078c2f7ff5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d02cf2496944f959f15c078c2f7ff5a" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 33659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8e295f235974861a5124ddc7d691d58" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00218ac386364021825bd67e6c27101e" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 41185 } health_report { overall_health: UNKNOWN } } }
I20260619 02:39:38.722769 7701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:38.720435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53764 (local address 127.5.241.66:33659)
0619 02:39:38.720494 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:38.720496 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:38.720504 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:39:38.720550 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:38.720550 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:38.720564 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:38.720617 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:39:38.720747 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.721193 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.722088 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.722282 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.722333 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:38.722483 (+ 150us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:39:38.722563 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:39:38.722616 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:38.722663 (+ 47us) server_negotiation.cc:299] Negotiation successful
0619 02:39:38.722682 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 02:39:38.724472 7357 raft_consensus.cc:1261] T 1a33c6f14953460aaea887317b805e2d P d8e295f235974861a5124ddc7d691d58 [term 1 FOLLOWER]: Refusing update from remote peer 8d02cf2496944f959f15c078c2f7ff5a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:39:38.724474 7637 raft_consensus.cc:1261] T 1a33c6f14953460aaea887317b805e2d P 00218ac386364021825bd67e6c27101e [term 1 FOLLOWER]: Refusing update from remote peer 8d02cf2496944f959f15c078c2f7ff5a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:39:38.724794 7716 consensus_queue.cc:1035] T 1a33c6f14953460aaea887317b805e2d P 8d02cf2496944f959f15c078c2f7ff5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00218ac386364021825bd67e6c27101e" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 41185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:39:38.724963 7714 consensus_queue.cc:1035] T 1a33c6f14953460aaea887317b805e2d P 8d02cf2496944f959f15c078c2f7ff5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8e295f235974861a5124ddc7d691d58" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:39:38.727190 7719 mvcc.cc:205] Tried to move back new op lower bound from 7298403445652856832 to 7298403445601271808. Current Snapshot: MvccSnapshot[applied={T|T < 7298403445652856832}]
I20260619 02:39:38.728295 7721 mvcc.cc:205] Tried to move back new op lower bound from 7298403445652856832 to 7298403445601271808. Current Snapshot: MvccSnapshot[applied={T|T < 7298403445652856832}]
I20260619 02:39:38.731264 7200 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:39:38.731345 7200 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:39:38.732375 7200 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 87fa180e998d416da04956de2d1bd04c: Sending DeleteTablet for 3 replicas of tablet 1a33c6f14953460aaea887317b805e2d
I20260619 02:39:38.732718 7477 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a33c6f14953460aaea887317b805e2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:39:38 UTC) from {username='kudu', principal='kudu/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:53756
I20260619 02:39:38.732733 7617 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a33c6f14953460aaea887317b805e2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:39:38 UTC) from {username='kudu', principal='kudu/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:33182
I20260619 02:39:38.732923 7337 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a33c6f14953460aaea887317b805e2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:39:38 UTC) from {username='kudu', principal='kudu/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:55084
I20260619 02:39:38.732934 7729 tablet_replica.cc:330] T 1a33c6f14953460aaea887317b805e2d P 00218ac386364021825bd67e6c27101e: stopping tablet replica
I20260619 02:39:38.733069 7729 raft_consensus.cc:2229] T 1a33c6f14953460aaea887317b805e2d P 00218ac386364021825bd67e6c27101e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:39:38.733110 7731 tablet_replica.cc:330] T 1a33c6f14953460aaea887317b805e2d P d8e295f235974861a5124ddc7d691d58: stopping tablet replica
I20260619 02:39:38.733206 7731 raft_consensus.cc:2229] T 1a33c6f14953460aaea887317b805e2d P d8e295f235974861a5124ddc7d691d58 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:39:38.733222 7729 raft_consensus.cc:2258] T 1a33c6f14953460aaea887317b805e2d P 00218ac386364021825bd67e6c27101e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:39:38.733219 7200 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50832:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260619 02:39:38.733311 7731 raft_consensus.cc:2258] T 1a33c6f14953460aaea887317b805e2d P d8e295f235974861a5124ddc7d691d58 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260619 02:39:38.733355 7200 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:39:38.733501 7730 tablet_replica.cc:330] T 1a33c6f14953460aaea887317b805e2d P 8d02cf2496944f959f15c078c2f7ff5a: stopping tablet replica
I20260619 02:39:38.733598 7730 raft_consensus.cc:2229] T 1a33c6f14953460aaea887317b805e2d P 8d02cf2496944f959f15c078c2f7ff5a [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:39:38.733716 7730 raft_consensus.cc:2258] T 1a33c6f14953460aaea887317b805e2d P 8d02cf2496944f959f15c078c2f7ff5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:39:38.733976 7731 ts_tablet_manager.cc:1905] T 1a33c6f14953460aaea887317b805e2d P d8e295f235974861a5124ddc7d691d58: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:39:38.734371 7729 ts_tablet_manager.cc:1905] T 1a33c6f14953460aaea887317b805e2d P 00218ac386364021825bd67e6c27101e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:39:38.734674 7617 tablet_service.cc:1467] Processing CreateTablet for tablet b6fcbc97c9224b8dba32fdf95a81e20e (DEFAULT_TABLE table=test-table [id=673ebe0b27ac450088ad26c13bfd956b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:39:38.734814 7617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6fcbc97c9224b8dba32fdf95a81e20e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:39:38.734853 7477 tablet_service.cc:1467] Processing CreateTablet for tablet b6fcbc97c9224b8dba32fdf95a81e20e (DEFAULT_TABLE table=test-table [id=673ebe0b27ac450088ad26c13bfd956b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:39:38.734951 7477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6fcbc97c9224b8dba32fdf95a81e20e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:39:38.736192 7706 tablet_bootstrap.cc:492] T b6fcbc97c9224b8dba32fdf95a81e20e P 00218ac386364021825bd67e6c27101e: Bootstrap starting.
I20260619 02:39:38.736245 7708 tablet_bootstrap.cc:492] T b6fcbc97c9224b8dba32fdf95a81e20e P 8d02cf2496944f959f15c078c2f7ff5a: Bootstrap starting.
I20260619 02:39:38.736645 7708 tablet_bootstrap.cc:654] T b6fcbc97c9224b8dba32fdf95a81e20e P 8d02cf2496944f959f15c078c2f7ff5a: Neither blocks nor log segments found. Creating new log.
I20260619 02:39:38.736690 7706 tablet_bootstrap.cc:654] T b6fcbc97c9224b8dba32fdf95a81e20e P 00218ac386364021825bd67e6c27101e: Neither blocks nor log segments found. Creating new log.
I20260619 02:39:38.737211 7337 tablet_service.cc:1467] Processing CreateTablet for tablet b6fcbc97c9224b8dba32fdf95a81e20e (DEFAULT_TABLE table=test-table [id=673ebe0b27ac450088ad26c13bfd956b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:39:38.737313 7337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6fcbc97c9224b8dba32fdf95a81e20e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:39:38.737782 7729 ts_tablet_manager.cc:1918] T 1a33c6f14953460aaea887317b805e2d P 00218ac386364021825bd67e6c27101e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:39:38.737843 7729 log.cc:1196] T 1a33c6f14953460aaea887317b805e2d P 00218ac386364021825bd67e6c27101e: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-2/wal/wals/1a33c6f14953460aaea887317b805e2d
I20260619 02:39:38.738090 7729 ts_tablet_manager.cc:1939] T 1a33c6f14953460aaea887317b805e2d P 00218ac386364021825bd67e6c27101e: Deleting consensus metadata
I20260619 02:39:38.738636 7706 tablet_bootstrap.cc:492] T b6fcbc97c9224b8dba32fdf95a81e20e P 00218ac386364021825bd67e6c27101e: No bootstrap required, opened a new log
I20260619 02:39:38.738682 7706 ts_tablet_manager.cc:1397] T b6fcbc97c9224b8dba32fdf95a81e20e P 00218ac386364021825bd67e6c27101e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260619 02:39:38.738683 7707 tablet_bootstrap.cc:492] T b6fcbc97c9224b8dba32fdf95a81e20e P d8e295f235974861a5124ddc7d691d58: Bootstrap starting.
I20260619 02:39:38.738814 7187 catalog_manager.cc:4841] TS 00218ac386364021825bd67e6c27101e (127.5.241.67:41185): tablet 1a33c6f14953460aaea887317b805e2d (table test-table [id=b3be3dbaff5147c8a99186e2decd85fd]) successfully deleted
I20260619 02:39:38.738852 7706 raft_consensus.cc:348] T b6fcbc97c9224b8dba32fdf95a81e20e P 00218ac386364021825bd67e6c27101e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e295f235974861a5124ddc7d691d58" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46725 } } peers { permanent_uuid: "8d02cf2496944f959f15c078c2f7ff5a" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 33659 } } peers { permanent_uuid: "00218ac386364021825bd67e6c27101e" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 41185 } }
I20260619 02:39:38.738914 7706 raft_consensus.cc:374] T b6fcbc97c9224b8dba32fdf95a81e20e P 00218ac386364021825bd67e6c27101e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:39:38.738937 7706 raft_consensus.cc:729] T b6fcbc97c9224b8dba32fdf95a81e20e P 00218ac386364021825bd67e6c27101e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00218ac386364021825bd67e6c27101e, State: Initialized, Role: FOLLOWER
I20260619 02:39:38.738976 7706 consensus_queue.cc:260] T b6fcbc97c9224b8dba32fdf95a81e20e P 00218ac386364021825bd67e6c27101e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e295f235974861a5124ddc7d691d58" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46725 } } peers { permanent_uuid: "8d02cf2496944f959f15c078c2f7ff5a" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 33659 } } peers { permanent_uuid: "00218ac386364021825bd67e6c27101e" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 41185 } }
I20260619 02:39:38.739055 7706 ts_tablet_manager.cc:1428] T b6fcbc97c9224b8dba32fdf95a81e20e P 00218ac386364021825bd67e6c27101e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:38.739114 7707 tablet_bootstrap.cc:654] T b6fcbc97c9224b8dba32fdf95a81e20e P d8e295f235974861a5124ddc7d691d58: Neither blocks nor log segments found. Creating new log.
I20260619 02:39:38.739228 7730 ts_tablet_manager.cc:1905] T 1a33c6f14953460aaea887317b805e2d P 8d02cf2496944f959f15c078c2f7ff5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:39:38.739720 7707 tablet_bootstrap.cc:492] T b6fcbc97c9224b8dba32fdf95a81e20e P d8e295f235974861a5124ddc7d691d58: No bootstrap required, opened a new log
I20260619 02:39:38.739765 7707 ts_tablet_manager.cc:1397] T b6fcbc97c9224b8dba32fdf95a81e20e P d8e295f235974861a5124ddc7d691d58: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:39:38.740275 7712 raft_consensus.cc:482] T b6fcbc97c9224b8dba32fdf95a81e20e P 00218ac386364021825bd67e6c27101e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:39:38.740325 7712 raft_consensus.cc:504] T b6fcbc97c9224b8dba32fdf95a81e20e P 00218ac386364021825bd67e6c27101e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e295f235974861a5124ddc7d691d58" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46725 } } peers { permanent_uuid: "8d02cf2496944f959f15c078c2f7ff5a" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 33659 } } peers { permanent_uuid: "00218ac386364021825bd67e6c27101e" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 41185 } }
I20260619 02:39:38.740330 7707 raft_consensus.cc:348] T b6fcbc97c9224b8dba32fdf95a81e20e P d8e295f235974861a5124ddc7d691d58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e295f235974861a5124ddc7d691d58" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46725 } } peers { permanent_uuid: "8d02cf2496944f959f15c078c2f7ff5a" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 33659 } } peers { permanent_uuid: "00218ac386364021825bd67e6c27101e" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 41185 } }
I20260619 02:39:38.740407 7707 raft_consensus.cc:374] T b6fcbc97c9224b8dba32fdf95a81e20e P d8e295f235974861a5124ddc7d691d58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:39:38.740434 7707 raft_consensus.cc:729] T b6fcbc97c9224b8dba32fdf95a81e20e P d8e295f235974861a5124ddc7d691d58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8e295f235974861a5124ddc7d691d58, State: Initialized, Role: FOLLOWER
I20260619 02:39:38.740484 7707 consensus_queue.cc:260] T b6fcbc97c9224b8dba32fdf95a81e20e P d8e295f235974861a5124ddc7d691d58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e295f235974861a5124ddc7d691d58" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46725 } } peers { permanent_uuid: "8d02cf2496944f959f15c078c2f7ff5a" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 33659 } } peers { permanent_uuid: "00218ac386364021825bd67e6c27101e" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 41185 } }
I20260619 02:39:38.740571 7707 ts_tablet_manager.cc:1428] T b6fcbc97c9224b8dba32fdf95a81e20e P d8e295f235974861a5124ddc7d691d58: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:39:38.740557 7712 leader_election.cc:288] T b6fcbc97c9224b8dba32fdf95a81e20e P 00218ac386364021825bd67e6c27101e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8e295f235974861a5124ddc7d691d58 (127.5.241.65:46725), 8d02cf2496944f959f15c078c2f7ff5a (127.5.241.66:33659)
I20260619 02:39:38.741868 7731 ts_tablet_manager.cc:1918] T 1a33c6f14953460aaea887317b805e2d P d8e295f235974861a5124ddc7d691d58: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:39:38.741935 7731 log.cc:1196] T 1a33c6f14953460aaea887317b805e2d P d8e295f235974861a5124ddc7d691d58: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-0/wal/wals/1a33c6f14953460aaea887317b805e2d
I20260619 02:39:38.742962 7730 ts_tablet_manager.cc:1918] T 1a33c6f14953460aaea887317b805e2d P 8d02cf2496944f959f15c078c2f7ff5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:39:38.743031 7730 log.cc:1196] T 1a33c6f14953460aaea887317b805e2d P 8d02cf2496944f959f15c078c2f7ff5a: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-1/wal/wals/1a33c6f14953460aaea887317b805e2d
I20260619 02:39:38.743314 7730 ts_tablet_manager.cc:1939] T 1a33c6f14953460aaea887317b805e2d P 8d02cf2496944f959f15c078c2f7ff5a: Deleting consensus metadata
I20260619 02:39:38.743682 7731 ts_tablet_manager.cc:1939] T 1a33c6f14953460aaea887317b805e2d P d8e295f235974861a5124ddc7d691d58: Deleting consensus metadata
I20260619 02:39:38.743680 7187 catalog_manager.cc:4841] TS 8d02cf2496944f959f15c078c2f7ff5a (127.5.241.66:33659): tablet 1a33c6f14953460aaea887317b805e2d (table test-table [id=b3be3dbaff5147c8a99186e2decd85fd]) successfully deleted
I20260619 02:39:38.744050 7187 catalog_manager.cc:4841] TS d8e295f235974861a5124ddc7d691d58 (127.5.241.65:46725): tablet 1a33c6f14953460aaea887317b805e2d (table test-table [id=b3be3dbaff5147c8a99186e2decd85fd]) successfully deleted
I20260619 02:39:38.744427 7708 tablet_bootstrap.cc:492] T b6fcbc97c9224b8dba32fdf95a81e20e P 8d02cf2496944f959f15c078c2f7ff5a: No bootstrap required, opened a new log
I20260619 02:39:38.744472 7708 ts_tablet_manager.cc:1397] T b6fcbc97c9224b8dba32fdf95a81e20e P 8d02cf2496944f959f15c078c2f7ff5a: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20260619 02:39:38.744604 7708 raft_consensus.cc:348] T b6fcbc97c9224b8dba32fdf95a81e20e P 8d02cf2496944f959f15c078c2f7ff5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e295f235974861a5124ddc7d691d58" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46725 } } peers { permanent_uuid: "8d02cf2496944f959f15c078c2f7ff5a" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 33659 } } peers { permanent_uuid: "00218ac386364021825bd67e6c27101e" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 41185 } }
I20260619 02:39:38.744635 7576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:38.740771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:46725 (local address 127.5.241.67:47735)
0619 02:39:38.740958 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:38.740969 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:38.741008 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:38.741208 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:38.741210 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:38.741212 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:39:38.741274 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:38.741276 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.742170 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.742171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:38.743395 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:38.743398 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.744363 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.744364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:38.744458 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:38.744463 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 02:39:38.744512 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 02:39:38.744554 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170}
I20260619 02:39:38.744658 7708 raft_consensus.cc:374] T b6fcbc97c9224b8dba32fdf95a81e20e P 8d02cf2496944f959f15c078c2f7ff5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:39:38.744689 7708 raft_consensus.cc:729] T b6fcbc97c9224b8dba32fdf95a81e20e P 8d02cf2496944f959f15c078c2f7ff5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d02cf2496944f959f15c078c2f7ff5a, State: Initialized, Role: FOLLOWER
I20260619 02:39:38.744735 7708 consensus_queue.cc:260] T b6fcbc97c9224b8dba32fdf95a81e20e P 8d02cf2496944f959f15c078c2f7ff5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e295f235974861a5124ddc7d691d58" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46725 } } peers { permanent_uuid: "8d02cf2496944f959f15c078c2f7ff5a" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 33659 } } peers { permanent_uuid: "00218ac386364021825bd67e6c27101e" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 41185 } }
I20260619 02:39:38.744835 7708 ts_tablet_manager.cc:1428] T b6fcbc97c9224b8dba32fdf95a81e20e P 8d02cf2496944f959f15c078c2f7ff5a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:38.744899 7703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:38.740956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:47735 (local address 127.5.241.65:46725)
0619 02:39:38.740991 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:38.740993 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:38.741001 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:39:38.741069 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:38.741070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:38.741079 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:38.741129 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:39:38.741362 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.742074 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.743495 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.744040 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.744108 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:38.744117 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:38.744802 (+ 685us) server_negotiation.cc:299] Negotiation successful
0619 02:39:38.744848 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 02:39:38.745065 7357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6fcbc97c9224b8dba32fdf95a81e20e" candidate_uuid: "00218ac386364021825bd67e6c27101e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8e295f235974861a5124ddc7d691d58" is_pre_election: true
I20260619 02:39:38.745149 7357 raft_consensus.cc:2454] T b6fcbc97c9224b8dba32fdf95a81e20e P d8e295f235974861a5124ddc7d691d58 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00218ac386364021825bd67e6c27101e in term 0.
I20260619 02:39:38.745301 7571 leader_election.cc:302] T b6fcbc97c9224b8dba32fdf95a81e20e P 00218ac386364021825bd67e6c27101e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00218ac386364021825bd67e6c27101e, d8e295f235974861a5124ddc7d691d58; no voters:
I20260619 02:39:38.745474 7712 raft_consensus.cc:2781] T b6fcbc97c9224b8dba32fdf95a81e20e P 00218ac386364021825bd67e6c27101e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:39:38.745517 7712 raft_consensus.cc:482] T b6fcbc97c9224b8dba32fdf95a81e20e P 00218ac386364021825bd67e6c27101e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:39:38.745537 7712 raft_consensus.cc:3037] T b6fcbc97c9224b8dba32fdf95a81e20e P 00218ac386364021825bd67e6c27101e [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:39:38.746114 7701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:38.741128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:47749 (local address 127.5.241.66:33659)
0619 02:39:38.741166 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:38.741168 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:38.742271 (+ 1103us) server_negotiation.cc:407] Connection header received
0619 02:39:38.742370 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:38.742370 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:38.742379 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:38.742406 (+ 27us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:39:38.742609 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.743334 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.745347 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.745795 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.745830 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:38.745835 (+ 5us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:38.746025 (+ 190us) server_negotiation.cc:299] Negotiation successful
0619 02:39:38.746057 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 02:39:38.746131 7735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:38.740852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.66:33659 (local address 127.5.241.67:47749)
0619 02:39:38.742213 (+ 1361us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:38.742224 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:38.742298 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:38.742462 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:38.742463 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:38.742464 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:39:38.742541 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:38.742542 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.743437 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.743437 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:38.744894 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:38.744897 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.745880 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.745881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:38.745938 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:38.745941 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:39:38.746021 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 02:39:38.746059 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1347,"thread_start_us":43,"threads_started":1}
I20260619 02:39:38.746245 7712 raft_consensus.cc:504] T b6fcbc97c9224b8dba32fdf95a81e20e P 00218ac386364021825bd67e6c27101e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e295f235974861a5124ddc7d691d58" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46725 } } peers { permanent_uuid: "8d02cf2496944f959f15c078c2f7ff5a" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 33659 } } peers { permanent_uuid: "00218ac386364021825bd67e6c27101e" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 41185 } }
I20260619 02:39:38.746357 7712 leader_election.cc:288] T b6fcbc97c9224b8dba32fdf95a81e20e P 00218ac386364021825bd67e6c27101e [CANDIDATE]: Term 1 election: Requested vote from peers d8e295f235974861a5124ddc7d691d58 (127.5.241.65:46725), 8d02cf2496944f959f15c078c2f7ff5a (127.5.241.66:33659)
I20260619 02:39:38.746394 7497 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6fcbc97c9224b8dba32fdf95a81e20e" candidate_uuid: "00218ac386364021825bd67e6c27101e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d02cf2496944f959f15c078c2f7ff5a" is_pre_election: true
I20260619 02:39:38.746507 7497 raft_consensus.cc:2454] T b6fcbc97c9224b8dba32fdf95a81e20e P 8d02cf2496944f959f15c078c2f7ff5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00218ac386364021825bd67e6c27101e in term 0.
I20260619 02:39:38.746529 7357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6fcbc97c9224b8dba32fdf95a81e20e" candidate_uuid: "00218ac386364021825bd67e6c27101e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8e295f235974861a5124ddc7d691d58"
I20260619 02:39:38.746584 7357 raft_consensus.cc:3037] T b6fcbc97c9224b8dba32fdf95a81e20e P d8e295f235974861a5124ddc7d691d58 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:39:38.746572 7496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6fcbc97c9224b8dba32fdf95a81e20e" candidate_uuid: "00218ac386364021825bd67e6c27101e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d02cf2496944f959f15c078c2f7ff5a"
I20260619 02:39:38.746636 7496 raft_consensus.cc:3037] T b6fcbc97c9224b8dba32fdf95a81e20e P 8d02cf2496944f959f15c078c2f7ff5a [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:39:38.747292 7357 raft_consensus.cc:2454] T b6fcbc97c9224b8dba32fdf95a81e20e P d8e295f235974861a5124ddc7d691d58 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00218ac386364021825bd67e6c27101e in term 1.
I20260619 02:39:38.747361 7496 raft_consensus.cc:2454] T b6fcbc97c9224b8dba32fdf95a81e20e P 8d02cf2496944f959f15c078c2f7ff5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00218ac386364021825bd67e6c27101e in term 1.
I20260619 02:39:38.747469 7571 leader_election.cc:302] T b6fcbc97c9224b8dba32fdf95a81e20e P 00218ac386364021825bd67e6c27101e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00218ac386364021825bd67e6c27101e, d8e295f235974861a5124ddc7d691d58; no voters:
I20260619 02:39:38.747609 7712 raft_consensus.cc:2781] T b6fcbc97c9224b8dba32fdf95a81e20e P 00218ac386364021825bd67e6c27101e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:39:38.747764 7712 raft_consensus.cc:686] T b6fcbc97c9224b8dba32fdf95a81e20e P 00218ac386364021825bd67e6c27101e [term 1 LEADER]: Becoming Leader. State: Replica: 00218ac386364021825bd67e6c27101e, State: Running, Role: LEADER
I20260619 02:39:38.747833 7712 consensus_queue.cc:237] T b6fcbc97c9224b8dba32fdf95a81e20e P 00218ac386364021825bd67e6c27101e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e295f235974861a5124ddc7d691d58" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46725 } } peers { permanent_uuid: "8d02cf2496944f959f15c078c2f7ff5a" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 33659 } } peers { permanent_uuid: "00218ac386364021825bd67e6c27101e" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 41185 } }
I20260619 02:39:38.748423 7200 catalog_manager.cc:5495] T b6fcbc97c9224b8dba32fdf95a81e20e P 00218ac386364021825bd67e6c27101e reported cstate change: term changed from 0 to 1, leader changed from <none> to 00218ac386364021825bd67e6c27101e (127.5.241.67). New cstate: current_term: 1 leader_uuid: "00218ac386364021825bd67e6c27101e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e295f235974861a5124ddc7d691d58" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d02cf2496944f959f15c078c2f7ff5a" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 33659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00218ac386364021825bd67e6c27101e" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 41185 } health_report { overall_health: HEALTHY } } }
Jun 19 02:39:38 dist-test-slave-zpbg krb5kdc[7155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836778, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:39:38.791319 7260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:38.783634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50846 (local address 127.5.241.126:39901)
0619 02:39:38.783706 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:38.783708 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:38.783792 (+ 84us) server_negotiation.cc:407] Connection header received
0619 02:39:38.783832 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:38.783833 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:38.783844 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:38.783921 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:38.784377 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.785150 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.785857 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.786158 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.786239 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:38.788523 (+ 2284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:38.788525 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:38.788525 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:38.788532 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:38.790091 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:38.790480 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:38.790481 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:38.790482 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:38.790542 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:38.790761 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:38.790762 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:38.790762 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:38.790908 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:38.790973 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:38.791128 (+ 155us) server_negotiation.cc:299] Negotiation successful
0619 02:39:38.791222 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 02:39:38.791433 7743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:38.783549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39901 (local address 127.0.0.1:50846)
0619 02:39:38.783714 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:38.783732 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:38.783790 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:38.783933 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:38.783934 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:38.784253 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:39:38.784300 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:38.784302 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.785263 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.785263 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:38.785781 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:38.785783 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.786258 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.786259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:38.786309 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:38.786931 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:39:38.786935 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:39:38.788442 (+ 1507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:39:38.790185 (+ 1743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:38.790193 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:38.790195 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:38.790402 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:38.790627 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:38.790628 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:38.790628 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:38.790692 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:38.790979 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:39:38.790988 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:39:38.791029 (+ 41us) client_negotiation.cc:769] Sending connection context
0619 02:39:38.791086 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 02:39:38.791296 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":86,"thread_start_us":82,"threads_started":1}
I20260619 02:39:38.793520 7200 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:50846:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 02:39:38.793728 7200 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:39:38.795234 7337 tablet_service.cc:1467] Processing CreateTablet for tablet 136e5bfd6fd84c5eb7e9dfaba7536ca5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8f58c42679114370ba1f788fb8cc8eba]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 02:39:38.795282 7617 tablet_service.cc:1467] Processing CreateTablet for tablet 136e5bfd6fd84c5eb7e9dfaba7536ca5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8f58c42679114370ba1f788fb8cc8eba]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 02:39:38.795397 7617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 136e5bfd6fd84c5eb7e9dfaba7536ca5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:39:38.795398 7337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 136e5bfd6fd84c5eb7e9dfaba7536ca5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:39:38.796891 7706 tablet_bootstrap.cc:492] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P 00218ac386364021825bd67e6c27101e: Bootstrap starting.
I20260619 02:39:38.796964 7477 tablet_service.cc:1467] Processing CreateTablet for tablet 136e5bfd6fd84c5eb7e9dfaba7536ca5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8f58c42679114370ba1f788fb8cc8eba]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 02:39:38.797092 7477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 136e5bfd6fd84c5eb7e9dfaba7536ca5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:39:38.797343 7706 tablet_bootstrap.cc:654] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P 00218ac386364021825bd67e6c27101e: Neither blocks nor log segments found. Creating new log.
I20260619 02:39:38.797708 7707 tablet_bootstrap.cc:492] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P d8e295f235974861a5124ddc7d691d58: Bootstrap starting.
I20260619 02:39:38.798034 7706 tablet_bootstrap.cc:492] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P 00218ac386364021825bd67e6c27101e: No bootstrap required, opened a new log
I20260619 02:39:38.798074 7706 ts_tablet_manager.cc:1397] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P 00218ac386364021825bd67e6c27101e: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 02:39:38.798084 7707 tablet_bootstrap.cc:654] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P d8e295f235974861a5124ddc7d691d58: Neither blocks nor log segments found. Creating new log.
I20260619 02:39:38.798243 7706 raft_consensus.cc:348] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P 00218ac386364021825bd67e6c27101e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e295f235974861a5124ddc7d691d58" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46725 } } peers { permanent_uuid: "8d02cf2496944f959f15c078c2f7ff5a" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 33659 } } peers { permanent_uuid: "00218ac386364021825bd67e6c27101e" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 41185 } }
I20260619 02:39:38.798312 7706 raft_consensus.cc:374] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P 00218ac386364021825bd67e6c27101e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:39:38.798331 7706 raft_consensus.cc:729] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P 00218ac386364021825bd67e6c27101e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00218ac386364021825bd67e6c27101e, State: Initialized, Role: FOLLOWER
I20260619 02:39:38.798377 7706 consensus_queue.cc:260] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P 00218ac386364021825bd67e6c27101e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e295f235974861a5124ddc7d691d58" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46725 } } peers { permanent_uuid: "8d02cf2496944f959f15c078c2f7ff5a" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 33659 } } peers { permanent_uuid: "00218ac386364021825bd67e6c27101e" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 41185 } }
I20260619 02:39:38.798475 7706 ts_tablet_manager.cc:1428] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P 00218ac386364021825bd67e6c27101e: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260619 02:39:38.798480 7708 tablet_bootstrap.cc:492] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P 8d02cf2496944f959f15c078c2f7ff5a: Bootstrap starting.
I20260619 02:39:38.798628 7736 tablet_replica.cc:439] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P 00218ac386364021825bd67e6c27101e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e295f235974861a5124ddc7d691d58" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46725 } } peers { permanent_uuid: "8d02cf2496944f959f15c078c2f7ff5a" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 33659 } } peers { permanent_uuid: "00218ac386364021825bd67e6c27101e" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 41185 } } }
I20260619 02:39:38.798728 7736 tablet_replica.cc:442] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P 00218ac386364021825bd67e6c27101e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:39:38.798789 7708 tablet_bootstrap.cc:654] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P 8d02cf2496944f959f15c078c2f7ff5a: Neither blocks nor log segments found. Creating new log.
I20260619 02:39:38.799651 7708 tablet_bootstrap.cc:492] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P 8d02cf2496944f959f15c078c2f7ff5a: No bootstrap required, opened a new log
I20260619 02:39:38.799702 7708 ts_tablet_manager.cc:1397] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P 8d02cf2496944f959f15c078c2f7ff5a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 02:39:38.799816 7708 raft_consensus.cc:348] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P 8d02cf2496944f959f15c078c2f7ff5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e295f235974861a5124ddc7d691d58" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46725 } } peers { permanent_uuid: "8d02cf2496944f959f15c078c2f7ff5a" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 33659 } } peers { permanent_uuid: "00218ac386364021825bd67e6c27101e" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 41185 } }
I20260619 02:39:38.799860 7708 raft_consensus.cc:374] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P 8d02cf2496944f959f15c078c2f7ff5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:39:38.799873 7708 raft_consensus.cc:729] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P 8d02cf2496944f959f15c078c2f7ff5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d02cf2496944f959f15c078c2f7ff5a, State: Initialized, Role: FOLLOWER
I20260619 02:39:38.799865 7707 tablet_bootstrap.cc:492] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P d8e295f235974861a5124ddc7d691d58: No bootstrap required, opened a new log
I20260619 02:39:38.799911 7707 ts_tablet_manager.cc:1397] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P d8e295f235974861a5124ddc7d691d58: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:39:38.799932 7708 consensus_queue.cc:260] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P 8d02cf2496944f959f15c078c2f7ff5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e295f235974861a5124ddc7d691d58" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46725 } } peers { permanent_uuid: "8d02cf2496944f959f15c078c2f7ff5a" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 33659 } } peers { permanent_uuid: "00218ac386364021825bd67e6c27101e" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 41185 } }
I20260619 02:39:38.800040 7708 ts_tablet_manager.cc:1428] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P 8d02cf2496944f959f15c078c2f7ff5a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:38.800055 7707 raft_consensus.cc:348] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P d8e295f235974861a5124ddc7d691d58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e295f235974861a5124ddc7d691d58" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46725 } } peers { permanent_uuid: "8d02cf2496944f959f15c078c2f7ff5a" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 33659 } } peers { permanent_uuid: "00218ac386364021825bd67e6c27101e" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 41185 } }
I20260619 02:39:38.800071 7724 tablet_replica.cc:439] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P 8d02cf2496944f959f15c078c2f7ff5a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e295f235974861a5124ddc7d691d58" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46725 } } peers { permanent_uuid: "8d02cf2496944f959f15c078c2f7ff5a" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 33659 } } peers { permanent_uuid: "00218ac386364021825bd67e6c27101e" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 41185 } } }
I20260619 02:39:38.800122 7707 raft_consensus.cc:374] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P d8e295f235974861a5124ddc7d691d58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:39:38.800141 7707 raft_consensus.cc:729] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P d8e295f235974861a5124ddc7d691d58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8e295f235974861a5124ddc7d691d58, State: Initialized, Role: FOLLOWER
I20260619 02:39:38.800133 7724 tablet_replica.cc:442] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P 8d02cf2496944f959f15c078c2f7ff5a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:39:38.800199 7707 consensus_queue.cc:260] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P d8e295f235974861a5124ddc7d691d58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e295f235974861a5124ddc7d691d58" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46725 } } peers { permanent_uuid: "8d02cf2496944f959f15c078c2f7ff5a" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 33659 } } peers { permanent_uuid: "00218ac386364021825bd67e6c27101e" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 41185 } }
I20260619 02:39:38.800277 7707 ts_tablet_manager.cc:1428] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P d8e295f235974861a5124ddc7d691d58: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 02:39:38.800408 7713 tablet_replica.cc:439] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P d8e295f235974861a5124ddc7d691d58: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e295f235974861a5124ddc7d691d58" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46725 } } peers { permanent_uuid: "8d02cf2496944f959f15c078c2f7ff5a" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 33659 } } peers { permanent_uuid: "00218ac386364021825bd67e6c27101e" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 41185 } } }
I20260619 02:39:38.800879 7713 tablet_replica.cc:442] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P d8e295f235974861a5124ddc7d691d58: This TxnStatusTablet replica's current role is: FOLLOWER
W20260619 02:39:38.838266 7404 tablet.cc:2369] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P d8e295f235974861a5124ddc7d691d58: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:39:38.893505 7713 raft_consensus.cc:482] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P d8e295f235974861a5124ddc7d691d58 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:39:38.893600 7713 raft_consensus.cc:504] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P d8e295f235974861a5124ddc7d691d58 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e295f235974861a5124ddc7d691d58" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46725 } } peers { permanent_uuid: "8d02cf2496944f959f15c078c2f7ff5a" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 33659 } } peers { permanent_uuid: "00218ac386364021825bd67e6c27101e" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 41185 } }
I20260619 02:39:38.893880 7713 leader_election.cc:288] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P d8e295f235974861a5124ddc7d691d58 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8d02cf2496944f959f15c078c2f7ff5a (127.5.241.66:33659), 00218ac386364021825bd67e6c27101e (127.5.241.67:41185)
I20260619 02:39:38.896950 7701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:38.894172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:60841 (local address 127.5.241.66:33659)
0619 02:39:38.894234 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:38.894236 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:38.894345 (+ 109us) server_negotiation.cc:407] Connection header received
0619 02:39:38.894383 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:38.894383 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:38.894393 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:38.894462 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:39:38.894654 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.895212 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.896181 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.896575 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.896668 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:38.896677 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:38.896837 (+ 160us) server_negotiation.cc:299] Negotiation successful
0619 02:39:38.896888 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 02:39:38.896961 7747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:38.894127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.66:33659 (local address 127.5.241.65:60841)
0619 02:39:38.894252 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:38.894265 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:38.894328 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:38.894490 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:38.894491 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:38.894494 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:39:38.894568 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:38.894570 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.895294 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.895295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:38.896079 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:38.896081 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.896675 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.896676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:38.896745 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:38.896748 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:39:38.896830 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 02:39:38.896874 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":95,"threads_started":1}
I20260619 02:39:38.897078 7705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:38.894422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:36201 (local address 127.5.241.67:41185)
0619 02:39:38.894476 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:38.894478 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:38.894488 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:39:38.894536 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:38.894537 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:38.894544 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:38.894617 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:39:38.894789 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.895256 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.896302 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.896666 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.896762 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:38.896770 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:38.896936 (+ 166us) server_negotiation.cc:299] Negotiation successful
0619 02:39:38.896975 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 02:39:38.897190 7496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "136e5bfd6fd84c5eb7e9dfaba7536ca5" candidate_uuid: "d8e295f235974861a5124ddc7d691d58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d02cf2496944f959f15c078c2f7ff5a" is_pre_election: true
I20260619 02:39:38.897281 7496 raft_consensus.cc:2454] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P 8d02cf2496944f959f15c078c2f7ff5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8e295f235974861a5124ddc7d691d58 in term 0.
W20260619 02:39:38.897308 7684 tablet.cc:2369] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P 00218ac386364021825bd67e6c27101e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:39:38.897343 7748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:38.894334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.67:41185 (local address 127.5.241.65:36201)
0619 02:39:38.894429 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:38.894438 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:38.894477 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:38.894620 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:38.894621 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:38.894623 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:39:38.894703 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:38.894704 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.895349 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.895350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:38.896211 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:38.896213 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.896765 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.896766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:38.896843 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:38.896847 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:39:38.897198 (+ 351us) client_negotiation.cc:240] Negotiation successful
0619 02:39:38.897251 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":39,"threads_started":1}
I20260619 02:39:38.897420 7291 leader_election.cc:302] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P d8e295f235974861a5124ddc7d691d58 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d02cf2496944f959f15c078c2f7ff5a, d8e295f235974861a5124ddc7d691d58; no voters:
I20260619 02:39:38.897562 7713 raft_consensus.cc:2781] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P d8e295f235974861a5124ddc7d691d58 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:39:38.897611 7713 raft_consensus.cc:482] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P d8e295f235974861a5124ddc7d691d58 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:39:38.897644 7713 raft_consensus.cc:3037] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P d8e295f235974861a5124ddc7d691d58 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:39:38.897744 7637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "136e5bfd6fd84c5eb7e9dfaba7536ca5" candidate_uuid: "d8e295f235974861a5124ddc7d691d58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00218ac386364021825bd67e6c27101e" is_pre_election: true
I20260619 02:39:38.897828 7637 raft_consensus.cc:2454] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P 00218ac386364021825bd67e6c27101e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8e295f235974861a5124ddc7d691d58 in term 0.
I20260619 02:39:38.898430 7713 raft_consensus.cc:504] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P d8e295f235974861a5124ddc7d691d58 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e295f235974861a5124ddc7d691d58" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46725 } } peers { permanent_uuid: "8d02cf2496944f959f15c078c2f7ff5a" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 33659 } } peers { permanent_uuid: "00218ac386364021825bd67e6c27101e" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 41185 } }
I20260619 02:39:38.898573 7713 leader_election.cc:288] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P d8e295f235974861a5124ddc7d691d58 [CANDIDATE]: Term 1 election: Requested vote from peers 8d02cf2496944f959f15c078c2f7ff5a (127.5.241.66:33659), 00218ac386364021825bd67e6c27101e (127.5.241.67:41185)
I20260619 02:39:38.898733 7496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "136e5bfd6fd84c5eb7e9dfaba7536ca5" candidate_uuid: "d8e295f235974861a5124ddc7d691d58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d02cf2496944f959f15c078c2f7ff5a"
I20260619 02:39:38.898833 7496 raft_consensus.cc:3037] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P 8d02cf2496944f959f15c078c2f7ff5a [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:39:38.898811 7637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "136e5bfd6fd84c5eb7e9dfaba7536ca5" candidate_uuid: "d8e295f235974861a5124ddc7d691d58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00218ac386364021825bd67e6c27101e"
I20260619 02:39:38.898887 7637 raft_consensus.cc:3037] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P 00218ac386364021825bd67e6c27101e [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:39:38.899412 7496 raft_consensus.cc:2454] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P 8d02cf2496944f959f15c078c2f7ff5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8e295f235974861a5124ddc7d691d58 in term 1.
I20260619 02:39:38.899564 7637 raft_consensus.cc:2454] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P 00218ac386364021825bd67e6c27101e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8e295f235974861a5124ddc7d691d58 in term 1.
I20260619 02:39:38.899582 7291 leader_election.cc:302] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P d8e295f235974861a5124ddc7d691d58 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d02cf2496944f959f15c078c2f7ff5a, d8e295f235974861a5124ddc7d691d58; no voters:
I20260619 02:39:38.899765 7713 raft_consensus.cc:2781] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P d8e295f235974861a5124ddc7d691d58 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:39:38.899906 7713 raft_consensus.cc:686] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P d8e295f235974861a5124ddc7d691d58 [term 1 LEADER]: Becoming Leader. State: Replica: d8e295f235974861a5124ddc7d691d58, State: Running, Role: LEADER
I20260619 02:39:38.899993 7713 consensus_queue.cc:237] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P d8e295f235974861a5124ddc7d691d58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e295f235974861a5124ddc7d691d58" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46725 } } peers { permanent_uuid: "8d02cf2496944f959f15c078c2f7ff5a" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 33659 } } peers { permanent_uuid: "00218ac386364021825bd67e6c27101e" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 41185 } }
I20260619 02:39:38.900296 7749 tablet_replica.cc:439] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P d8e295f235974861a5124ddc7d691d58: TxnStatusTablet state changed. Reason: New leader d8e295f235974861a5124ddc7d691d58. Latest consensus state: current_term: 1 leader_uuid: "d8e295f235974861a5124ddc7d691d58" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e295f235974861a5124ddc7d691d58" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46725 } } peers { permanent_uuid: "8d02cf2496944f959f15c078c2f7ff5a" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 33659 } } peers { permanent_uuid: "00218ac386364021825bd67e6c27101e" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 41185 } } }
I20260619 02:39:38.900389 7749 tablet_replica.cc:442] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P d8e295f235974861a5124ddc7d691d58: This TxnStatusTablet replica's current role is: LEADER
I20260619 02:39:38.900609 7751 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 02:39:38.900730 7751 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 02:39:38.900646 7199 catalog_manager.cc:5495] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P d8e295f235974861a5124ddc7d691d58 reported cstate change: term changed from 0 to 1, leader changed from <none> to d8e295f235974861a5124ddc7d691d58 (127.5.241.65). New cstate: current_term: 1 leader_uuid: "d8e295f235974861a5124ddc7d691d58" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e295f235974861a5124ddc7d691d58" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8d02cf2496944f959f15c078c2f7ff5a" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 33659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00218ac386364021825bd67e6c27101e" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 41185 } health_report { overall_health: UNKNOWN } } }
I20260619 02:39:38.919032 7743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:38.916320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:46725 (local address 127.0.0.1:55096)
0619 02:39:38.916392 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:38.916403 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:38.916457 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:38.916616 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:38.916617 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:38.916618 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:39:38.916663 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:38.916665 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.917534 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.917534 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:38.918085 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:38.918087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.918532 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.918533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:38.918578 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:38.918592 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:39:38.918895 (+ 303us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:39:38.918902 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 02:39:38.918934 (+ 32us) client_negotiation.cc:240] Negotiation successful
0619 02:39:38.918973 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 02:39:38.919040 7703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:38.916374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55096 (local address 127.5.241.65:46725)
0619 02:39:38.916413 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:38.916415 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:38.916443 (+ 28us) server_negotiation.cc:407] Connection header received
0619 02:39:38.916517 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:38.916518 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:38.916536 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:38.916614 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:39:38.916730 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.917463 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.918158 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.918455 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.918518 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:38.918677 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:39:38.918815 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:39:38.918897 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:38.918939 (+ 42us) server_negotiation.cc:299] Negotiation successful
0619 02:39:38.918957 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 02:39:38.920300 7637 raft_consensus.cc:1261] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P 00218ac386364021825bd67e6c27101e [term 1 FOLLOWER]: Refusing update from remote peer d8e295f235974861a5124ddc7d691d58: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:39:38.920393 7496 raft_consensus.cc:1261] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P 8d02cf2496944f959f15c078c2f7ff5a [term 1 FOLLOWER]: Refusing update from remote peer d8e295f235974861a5124ddc7d691d58: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:39:38.920701 7713 consensus_queue.cc:1035] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P d8e295f235974861a5124ddc7d691d58 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00218ac386364021825bd67e6c27101e" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 41185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:39:38.920830 7749 consensus_queue.cc:1035] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P d8e295f235974861a5124ddc7d691d58 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d02cf2496944f959f15c078c2f7ff5a" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 33659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:39:38.921378 7736 tablet_replica.cc:439] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P 00218ac386364021825bd67e6c27101e: TxnStatusTablet state changed. Reason: New leader d8e295f235974861a5124ddc7d691d58. Latest consensus state: current_term: 1 leader_uuid: "d8e295f235974861a5124ddc7d691d58" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e295f235974861a5124ddc7d691d58" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46725 } } peers { permanent_uuid: "8d02cf2496944f959f15c078c2f7ff5a" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 33659 } } peers { permanent_uuid: "00218ac386364021825bd67e6c27101e" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 41185 } } }
I20260619 02:39:38.921404 7724 tablet_replica.cc:439] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P 8d02cf2496944f959f15c078c2f7ff5a: TxnStatusTablet state changed. Reason: New leader d8e295f235974861a5124ddc7d691d58. Latest consensus state: current_term: 1 leader_uuid: "d8e295f235974861a5124ddc7d691d58" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e295f235974861a5124ddc7d691d58" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46725 } } peers { permanent_uuid: "8d02cf2496944f959f15c078c2f7ff5a" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 33659 } } peers { permanent_uuid: "00218ac386364021825bd67e6c27101e" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 41185 } } }
I20260619 02:39:38.921473 7736 tablet_replica.cc:442] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P 00218ac386364021825bd67e6c27101e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:39:38.921484 7724 tablet_replica.cc:442] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P 8d02cf2496944f959f15c078c2f7ff5a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:39:38.922202 7749 tablet_replica.cc:439] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P d8e295f235974861a5124ddc7d691d58: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d8e295f235974861a5124ddc7d691d58" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e295f235974861a5124ddc7d691d58" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46725 } } peers { permanent_uuid: "8d02cf2496944f959f15c078c2f7ff5a" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 33659 } } peers { permanent_uuid: "00218ac386364021825bd67e6c27101e" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 41185 } } }
I20260619 02:39:38.922309 7749 tablet_replica.cc:442] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P d8e295f235974861a5124ddc7d691d58: This TxnStatusTablet replica's current role is: LEADER
I20260619 02:39:38.922554 7736 tablet_replica.cc:439] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P 00218ac386364021825bd67e6c27101e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d8e295f235974861a5124ddc7d691d58" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e295f235974861a5124ddc7d691d58" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46725 } } peers { permanent_uuid: "8d02cf2496944f959f15c078c2f7ff5a" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 33659 } } peers { permanent_uuid: "00218ac386364021825bd67e6c27101e" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 41185 } } }
I20260619 02:39:38.922642 7736 tablet_replica.cc:442] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P 00218ac386364021825bd67e6c27101e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:39:38.923642 7713 tablet_replica.cc:439] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P d8e295f235974861a5124ddc7d691d58: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d8e295f235974861a5124ddc7d691d58" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e295f235974861a5124ddc7d691d58" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46725 } } peers { permanent_uuid: "8d02cf2496944f959f15c078c2f7ff5a" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 33659 } } peers { permanent_uuid: "00218ac386364021825bd67e6c27101e" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 41185 } } }
I20260619 02:39:38.923734 7713 tablet_replica.cc:442] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P d8e295f235974861a5124ddc7d691d58: This TxnStatusTablet replica's current role is: LEADER
I20260619 02:39:38.924105 7724 tablet_replica.cc:439] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P 8d02cf2496944f959f15c078c2f7ff5a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d8e295f235974861a5124ddc7d691d58" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e295f235974861a5124ddc7d691d58" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46725 } } peers { permanent_uuid: "8d02cf2496944f959f15c078c2f7ff5a" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 33659 } } peers { permanent_uuid: "00218ac386364021825bd67e6c27101e" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 41185 } } }
I20260619 02:39:38.924207 7724 tablet_replica.cc:442] T 136e5bfd6fd84c5eb7e9dfaba7536ca5 P 8d02cf2496944f959f15c078c2f7ff5a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:39:38.927112 7705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:38.924367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33190 (local address 127.5.241.67:41185)
0619 02:39:38.924412 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:38.924414 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:38.924421 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:39:38.924454 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:38.924455 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:38.924477 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:38.924541 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:39:38.924746 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.925540 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.926292 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.926592 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.926665 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:38.926804 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:39:38.926924 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:39:38.926997 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:38.927021 (+ 24us) server_negotiation.cc:299] Negotiation successful
0619 02:39:38.927041 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 02:39:38.931851 7703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:38.929494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:55739 (local address 127.5.241.65:46725)
0619 02:39:38.929535 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:38.929537 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:38.929545 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:39:38.929604 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:38.929604 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:38.929611 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:38.929651 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:39:38.929770 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.930213 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.931278 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.931450 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.931503 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:38.931676 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:39:38.931702 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:39:38.931740 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:38.931792 (+ 52us) server_negotiation.cc:299] Negotiation successful
0619 02:39:38.931805 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 02:39:38.931903 7689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:38.929421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:46725 (local address 127.5.241.67:55739)
0619 02:39:38.929502 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:38.929513 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:38.929549 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:38.929662 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:38.929663 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:38.929664 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:39:38.929715 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:38.929716 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.930290 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.930291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:38.931194 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:38.931196 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.931521 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.931522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:38.931586 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:38.931603 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:39:38.931741 (+ 138us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:39:38.931742 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 02:39:38.931791 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 02:39:38.931831 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260619 02:39:38.933929 7497 raft_consensus.cc:1261] T b6fcbc97c9224b8dba32fdf95a81e20e P 8d02cf2496944f959f15c078c2f7ff5a [term 1 FOLLOWER]: Refusing update from remote peer 00218ac386364021825bd67e6c27101e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:39:38.934026 7357 raft_consensus.cc:1261] T b6fcbc97c9224b8dba32fdf95a81e20e P d8e295f235974861a5124ddc7d691d58 [term 1 FOLLOWER]: Refusing update from remote peer 00218ac386364021825bd67e6c27101e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:39:38.934144 7736 consensus_queue.cc:1035] T b6fcbc97c9224b8dba32fdf95a81e20e P 00218ac386364021825bd67e6c27101e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d02cf2496944f959f15c078c2f7ff5a" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 33659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:39:38.934732 7736 consensus_queue.cc:1035] T b6fcbc97c9224b8dba32fdf95a81e20e P 00218ac386364021825bd67e6c27101e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8e295f235974861a5124ddc7d691d58" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:39:38.935931 7720 mvcc.cc:205] Tried to move back new op lower bound from 7298403446511894528 to 7298403445751689216. Current Snapshot: MvccSnapshot[applied={T|T < 7298403446511894528}]
I20260619 02:39:38.943661 7705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:38.940604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:52825 (local address 127.5.241.67:41185)
0619 02:39:38.940654 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:38.940656 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:38.940864 (+ 208us) server_negotiation.cc:407] Connection header received
0619 02:39:38.940911 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:38.940912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:38.940922 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:38.940985 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:39:38.941177 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.941688 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.942873 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.943174 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.943215 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:38.943466 (+ 251us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:39:38.943495 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:39:38.943548 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:38.943596 (+ 48us) server_negotiation.cc:299] Negotiation successful
0619 02:39:38.943611 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 02:39:38.943751 7763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:38.940434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.67:41185 (local address 127.5.241.65:52825)
0619 02:39:38.940789 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:38.940801 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:38.940861 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:38.941038 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:38.941039 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:38.941041 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:39:38.941110 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:38.941112 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.941756 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.941757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:38.942735 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:38.942738 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.943295 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.943296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:38.943371 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:38.943386 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:39:38.943543 (+ 157us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:39:38.943544 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 02:39:38.943594 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 02:39:38.943634 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":85,"threads_started":1}
I20260619 02:39:38.952778 7199 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:39:38.952895 7199 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:39:38.953518 7199 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 87fa180e998d416da04956de2d1bd04c: Sending DeleteTablet for 3 replicas of tablet b6fcbc97c9224b8dba32fdf95a81e20e
I20260619 02:39:38.953759 7337 tablet_service.cc:1514] Processing DeleteTablet for tablet b6fcbc97c9224b8dba32fdf95a81e20e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:39:38 UTC) from {username='kudu', principal='kudu/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:55084
I20260619 02:39:38.953823 7617 tablet_service.cc:1514] Processing DeleteTablet for tablet b6fcbc97c9224b8dba32fdf95a81e20e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:39:38 UTC) from {username='kudu', principal='kudu/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:33182
I20260619 02:39:38.953850 7731 tablet_replica.cc:330] T b6fcbc97c9224b8dba32fdf95a81e20e P d8e295f235974861a5124ddc7d691d58: stopping tablet replica
I20260619 02:39:38.953893 7731 raft_consensus.cc:2229] T b6fcbc97c9224b8dba32fdf95a81e20e P d8e295f235974861a5124ddc7d691d58 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:39:38.953904 7729 tablet_replica.cc:330] T b6fcbc97c9224b8dba32fdf95a81e20e P 00218ac386364021825bd67e6c27101e: stopping tablet replica
I20260619 02:39:38.953924 7731 raft_consensus.cc:2258] T b6fcbc97c9224b8dba32fdf95a81e20e P d8e295f235974861a5124ddc7d691d58 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:39:38.953943 7729 raft_consensus.cc:2229] T b6fcbc97c9224b8dba32fdf95a81e20e P 00218ac386364021825bd67e6c27101e [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:39:38.954031 7729 raft_consensus.cc:2258] T b6fcbc97c9224b8dba32fdf95a81e20e P 00218ac386364021825bd67e6c27101e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:39:38.954370 7477 tablet_service.cc:1514] Processing DeleteTablet for tablet b6fcbc97c9224b8dba32fdf95a81e20e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:39:38 UTC) from {username='kudu', principal='kudu/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:53756
I20260619 02:39:38.954460 7730 tablet_replica.cc:330] T b6fcbc97c9224b8dba32fdf95a81e20e P 8d02cf2496944f959f15c078c2f7ff5a: stopping tablet replica
I20260619 02:39:38.954504 7730 raft_consensus.cc:2229] T b6fcbc97c9224b8dba32fdf95a81e20e P 8d02cf2496944f959f15c078c2f7ff5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:39:38.954537 7730 raft_consensus.cc:2258] T b6fcbc97c9224b8dba32fdf95a81e20e P 8d02cf2496944f959f15c078c2f7ff5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:39:38.954559 7729 ts_tablet_manager.cc:1905] T b6fcbc97c9224b8dba32fdf95a81e20e P 00218ac386364021825bd67e6c27101e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:39:38.954923 7730 ts_tablet_manager.cc:1905] T b6fcbc97c9224b8dba32fdf95a81e20e P 8d02cf2496944f959f15c078c2f7ff5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:39:38.956444 7730 ts_tablet_manager.cc:1918] T b6fcbc97c9224b8dba32fdf95a81e20e P 8d02cf2496944f959f15c078c2f7ff5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:39:38.956504 7730 log.cc:1196] T b6fcbc97c9224b8dba32fdf95a81e20e P 8d02cf2496944f959f15c078c2f7ff5a: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-1/wal/wals/b6fcbc97c9224b8dba32fdf95a81e20e
I20260619 02:39:38.956840 7730 ts_tablet_manager.cc:1939] T b6fcbc97c9224b8dba32fdf95a81e20e P 8d02cf2496944f959f15c078c2f7ff5a: Deleting consensus metadata
I20260619 02:39:38.956884 7729 ts_tablet_manager.cc:1918] T b6fcbc97c9224b8dba32fdf95a81e20e P 00218ac386364021825bd67e6c27101e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:39:38.956938 7729 log.cc:1196] T b6fcbc97c9224b8dba32fdf95a81e20e P 00218ac386364021825bd67e6c27101e: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-2/wal/wals/b6fcbc97c9224b8dba32fdf95a81e20e
I20260619 02:39:38.957162 7729 ts_tablet_manager.cc:1939] T b6fcbc97c9224b8dba32fdf95a81e20e P 00218ac386364021825bd67e6c27101e: Deleting consensus metadata
I20260619 02:39:38.957410 7187 catalog_manager.cc:4841] TS 8d02cf2496944f959f15c078c2f7ff5a (127.5.241.66:33659): tablet b6fcbc97c9224b8dba32fdf95a81e20e (table test-table [id=673ebe0b27ac450088ad26c13bfd956b]) successfully deleted
I20260619 02:39:38.957489 7187 catalog_manager.cc:4841] TS 00218ac386364021825bd67e6c27101e (127.5.241.67:41185): tablet b6fcbc97c9224b8dba32fdf95a81e20e (table test-table [id=673ebe0b27ac450088ad26c13bfd956b]) successfully deleted
I20260619 02:39:38.957739 7731 ts_tablet_manager.cc:1905] T b6fcbc97c9224b8dba32fdf95a81e20e P d8e295f235974861a5124ddc7d691d58: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:39:38.959324 7731 ts_tablet_manager.cc:1918] T b6fcbc97c9224b8dba32fdf95a81e20e P d8e295f235974861a5124ddc7d691d58: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:39:38.959375 7731 log.cc:1196] T b6fcbc97c9224b8dba32fdf95a81e20e P d8e295f235974861a5124ddc7d691d58: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836772852275-6085-0/minicluster-data/ts-0/wal/wals/b6fcbc97c9224b8dba32fdf95a81e20e
I20260619 02:39:38.959575 7731 ts_tablet_manager.cc:1939] T b6fcbc97c9224b8dba32fdf95a81e20e P d8e295f235974861a5124ddc7d691d58: Deleting consensus metadata
I20260619 02:39:38.959851 7187 catalog_manager.cc:4841] TS d8e295f235974861a5124ddc7d691d58 (127.5.241.65:46725): tablet b6fcbc97c9224b8dba32fdf95a81e20e (table test-table [id=673ebe0b27ac450088ad26c13bfd956b]) successfully deleted
Jun 19 02:39:38 dist-test-slave-zpbg krb5kdc[7155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836778, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.241.65@KRBTEST.COM
I20260619 02:39:38.973385 7703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:38.957005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55108 (local address 127.5.241.65:46725)
0619 02:39:38.957048 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:38.957050 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:38.962994 (+ 5944us) server_negotiation.cc:407] Connection header received
0619 02:39:38.963116 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:38.963118 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:38.963130 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:38.963202 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:38.963890 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.964665 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.965733 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.966041 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.966114 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:38.969649 (+ 3535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:38.969655 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:38.969656 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:38.969663 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:38.971655 (+ 1992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:38.972479 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:38.972483 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:38.972483 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:38.972551 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:38.972777 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:38.972778 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:38.972778 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:38.972950 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:38.973092 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:38.973189 (+ 97us) server_negotiation.cc:299] Negotiation successful
0619 02:39:38.973286 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260619 02:39:38.973891 7397 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55108
I20260619 02:39:38.982089 7260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:38.974849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50852 (local address 127.5.241.126:39901)
0619 02:39:38.974890 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:38.974892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:38.975060 (+ 168us) server_negotiation.cc:407] Connection header received
0619 02:39:38.975149 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:38.975149 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:38.975159 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:38.975188 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:38.975835 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.976549 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.977573 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:38.977859 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:38.977898 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:38.979369 (+ 1471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:38.979371 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:38.979371 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:38.979378 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:38.980923 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:38.981302 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:38.981303 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:38.981303 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:38.981356 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:38.981546 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:38.981547 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:38.981547 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:38.981661 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:38.981730 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:38.981926 (+ 196us) server_negotiation.cc:299] Negotiation successful
0619 02:39:38.982000 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260619 02:39:38.982961 7199 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50852
I20260619 02:39:38.983685 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 7272
I20260619 02:39:38.989558 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 7412
I20260619 02:39:38.995425 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 7552
I20260619 02:39:38.999693 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 7168
2026-06-19T02:39:39Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2160 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:39:39 dist-test-slave-zpbg krb5kdc[7772](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:39:39 dist-test-slave-zpbg krb5kdc[7772](info): set up 2 sockets
Jun 19 02:39:39 dist-test-slave-zpbg krb5kdc[7772](info): commencing operation
krb5kdc: starting...
W20260619 02:39:40.038411 6085 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Jun 19 02:39:40 dist-test-slave-zpbg krb5kdc[7772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:39:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:39:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.241.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
I20260619 02:39:40.157467 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:46063
--webserver_interface=127.5.241.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:45145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:46063
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:39:40.238191 7785 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:39:40.238411 7785 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:39:40.238435 7785 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:39:40.239634 7785 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:39:40.239682 7785 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 02:39:40.239697 7785 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:39:40.239710 7785 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:39:40.239722 7785 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:39:40.239735 7785 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:39:40.241226 7785 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:45145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:46063
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:46063
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.241.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7785
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:39:40.241460 7785 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:39:40.241675 7785 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:39:40.246678 7790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:39:40.246876 7785 server_base.cc:1034] running on GCE node
W20260619 02:39:40.246815 7793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:40.246683 7791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:39:40.247282 7785 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:39:40.247624 7785 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:39:40.248800 7785 hybrid_clock.cc:648] HybridClock initialized: now 1781836780248785 us; error 42 us; skew 500 ppm
Jun 19 02:39:40 dist-test-slave-zpbg krb5kdc[7772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836780, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:39:40.251488 7785 init.cc:377] Logged in from keytab as kudu/127.5.241.126@KRBTEST.COM (short username kudu)
I20260619 02:39:40.251940 7785 webserver.cc:460] Webserver started at http://127.5.241.126:34791/ using document root <none> and password file <none>
I20260619 02:39:40.252204 7785 fs_manager.cc:359] Metadata directory not provided
I20260619 02:39:40.252278 7785 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:39:40.252424 7785 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:39:40.253582 7785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/master-0/data/instance:
uuid: "61c08775450f47beb0d8ab94784cf7ab"
format_stamp: "Formatted at 2026-06-19 02:39:40 on dist-test-slave-zpbg"
server_key: "d9733db8a86c800490336e2bfacb3e02"
server_key_iv: "798c867281ebdaaa5876e0cea5f9be75"
server_key_version: "encryptionkey@0"
I20260619 02:39:40.253960 7785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/master-0/wal/instance:
uuid: "61c08775450f47beb0d8ab94784cf7ab"
format_stamp: "Formatted at 2026-06-19 02:39:40 on dist-test-slave-zpbg"
server_key: "d9733db8a86c800490336e2bfacb3e02"
server_key_iv: "798c867281ebdaaa5876e0cea5f9be75"
server_key_version: "encryptionkey@0"
I20260619 02:39:40.255407 7785 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:39:40.256230 7800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:40.256435 7785 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:39:40.256541 7785 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "61c08775450f47beb0d8ab94784cf7ab"
format_stamp: "Formatted at 2026-06-19 02:39:40 on dist-test-slave-zpbg"
server_key: "d9733db8a86c800490336e2bfacb3e02"
server_key_iv: "798c867281ebdaaa5876e0cea5f9be75"
server_key_version: "encryptionkey@0"
I20260619 02:39:40.256628 7785 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:39:40.279799 7785 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:39:40.281766 7785 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:39:40.281939 7785 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:39:40.286597 7785 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.126:46063
I20260619 02:39:40.286712 7862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.126:46063 every 8 connection(s)
I20260619 02:39:40.287035 7785 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:39:40.287717 7863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:39:40.290019 7863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61c08775450f47beb0d8ab94784cf7ab: Bootstrap starting.
I20260619 02:39:40.290616 7863 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 61c08775450f47beb0d8ab94784cf7ab: Neither blocks nor log segments found. Creating new log.
I20260619 02:39:40.290927 7863 log.cc:824] T 00000000000000000000000000000000 P 61c08775450f47beb0d8ab94784cf7ab: Log is configured to *not* fsync() on all Append() calls
I20260619 02:39:40.291697 7863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61c08775450f47beb0d8ab94784cf7ab: No bootstrap required, opened a new log
I20260619 02:39:40.292299 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 7785
I20260619 02:39:40.292385 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/master-0/wal/instance
I20260619 02:39:40.292562 6085 external_mini_cluster.cc:1428] Setting key f35917928246aa2eba194401d0e11428
I20260619 02:39:40.294482 7863 raft_consensus.cc:348] T 00000000000000000000000000000000 P 61c08775450f47beb0d8ab94784cf7ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61c08775450f47beb0d8ab94784cf7ab" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 46063 } }
I20260619 02:39:40.294662 7863 raft_consensus.cc:374] T 00000000000000000000000000000000 P 61c08775450f47beb0d8ab94784cf7ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:39:40.294704 7863 raft_consensus.cc:729] T 00000000000000000000000000000000 P 61c08775450f47beb0d8ab94784cf7ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61c08775450f47beb0d8ab94784cf7ab, State: Initialized, Role: FOLLOWER
I20260619 02:39:40.294915 7863 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61c08775450f47beb0d8ab94784cf7ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61c08775450f47beb0d8ab94784cf7ab" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 46063 } }
I20260619 02:39:40.295022 7863 raft_consensus.cc:388] T 00000000000000000000000000000000 P 61c08775450f47beb0d8ab94784cf7ab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:39:40.295059 7863 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61c08775450f47beb0d8ab94784cf7ab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:39:40.295117 7863 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61c08775450f47beb0d8ab94784cf7ab [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:39:40.295843 7863 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61c08775450f47beb0d8ab94784cf7ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61c08775450f47beb0d8ab94784cf7ab" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 46063 } }
I20260619 02:39:40.295939 7863 leader_election.cc:302] T 00000000000000000000000000000000 P 61c08775450f47beb0d8ab94784cf7ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 61c08775450f47beb0d8ab94784cf7ab; no voters:
I20260619 02:39:40.296149 7863 leader_election.cc:288] T 00000000000000000000000000000000 P 61c08775450f47beb0d8ab94784cf7ab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:39:40.296175 7868 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 61c08775450f47beb0d8ab94784cf7ab [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:39:40.296303 7868 raft_consensus.cc:686] T 00000000000000000000000000000000 P 61c08775450f47beb0d8ab94784cf7ab [term 1 LEADER]: Becoming Leader. State: Replica: 61c08775450f47beb0d8ab94784cf7ab, State: Running, Role: LEADER
I20260619 02:39:40.296391 7868 consensus_queue.cc:237] T 00000000000000000000000000000000 P 61c08775450f47beb0d8ab94784cf7ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61c08775450f47beb0d8ab94784cf7ab" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 46063 } }
I20260619 02:39:40.296446 7863 sys_catalog.cc:564] T 00000000000000000000000000000000 P 61c08775450f47beb0d8ab94784cf7ab [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:39:40.296734 7868 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61c08775450f47beb0d8ab94784cf7ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 61c08775450f47beb0d8ab94784cf7ab. Latest consensus state: current_term: 1 leader_uuid: "61c08775450f47beb0d8ab94784cf7ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61c08775450f47beb0d8ab94784cf7ab" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 46063 } } }
I20260619 02:39:40.296741 7870 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61c08775450f47beb0d8ab94784cf7ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "61c08775450f47beb0d8ab94784cf7ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61c08775450f47beb0d8ab94784cf7ab" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 46063 } } }
I20260619 02:39:40.296923 7868 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61c08775450f47beb0d8ab94784cf7ab [sys.catalog]: This master's current role is: LEADER
I20260619 02:39:40.296939 7870 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61c08775450f47beb0d8ab94784cf7ab [sys.catalog]: This master's current role is: LEADER
I20260619 02:39:40.297436 7874 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:39:40.298058 7874 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:39:40.299815 7874 catalog_manager.cc:1347] Generated new cluster ID: e75538e8a695480faf5a5bc11f8ad403
I20260619 02:39:40.299890 7874 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 02:39:40 dist-test-slave-zpbg krb5kdc[7772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:39:40.305580 7867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:40.293124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56818 (local address 127.5.241.126:46063)
0619 02:39:40.293320 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:40.293330 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:40.293545 (+ 215us) server_negotiation.cc:407] Connection header received
0619 02:39:40.294471 (+ 926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:40.294479 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:40.294534 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:40.294704 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:40.295547 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:40.296772 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:40.297939 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:40.298548 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:40.298660 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:40.301702 (+ 3042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:40.301706 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:40.301707 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:40.301718 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:40.303829 (+ 2111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:40.304247 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:40.304251 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:40.304252 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:40.304405 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:40.304611 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:40.304612 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:40.304612 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:40.304942 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:40.305108 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:40.305218 (+ 110us) server_negotiation.cc:299] Negotiation successful
0619 02:39:40.305371 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":57,"threads_started":1}
I20260619 02:39:40.307807 7874 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:39:40.308525 7874 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:39:40.311529 7874 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 61c08775450f47beb0d8ab94784cf7ab: Generated new TSK 0
I20260619 02:39:40.311697 7874 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.241.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab.
Entry for principal kudu/127.5.241.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab.
I20260619 02:39:40.343096 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.65:0
--local_ip_for_outbound_sockets=127.5.241.65
--webserver_interface=127.5.241.65
--webserver_port=0
--tserver_master_addrs=127.5.241.126:46063
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab
--principal=kudu/127.5.241.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:45145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:39:40.441185 7889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:39:40.441387 7889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:39:40.441413 7889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:39:40.442584 7889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:39:40.442628 7889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:39:40.442649 7889 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 02:39:40.442662 7889 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 02:39:40.442706 7889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.65
I20260619 02:39:40.444216 7889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:45145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab
--principal=kudu/127.5.241.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.241.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.241.126:46063
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7889
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.65
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:39:40.444468 7889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:39:40.444720 7889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:39:40.451382 7895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:40.451385 7894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:39:40.451606 7889 server_base.cc:1034] running on GCE node
W20260619 02:39:40.451756 7897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:39:40.451995 7889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:39:40.452275 7889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:39:40.453676 7889 hybrid_clock.cc:648] HybridClock initialized: now 1781836780453658 us; error 33 us; skew 500 ppm
Jun 19 02:39:40 dist-test-slave-zpbg krb5kdc[7772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836780, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:39:40.456553 7889 init.cc:377] Logged in from keytab as kudu/127.5.241.65@KRBTEST.COM (short username kudu)
I20260619 02:39:40.457108 7889 webserver.cc:460] Webserver started at http://127.5.241.65:37919/ using document root <none> and password file <none>
I20260619 02:39:40.457367 7889 fs_manager.cc:359] Metadata directory not provided
I20260619 02:39:40.457443 7889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:39:40.457597 7889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:39:40.458743 7889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-0/data/instance:
uuid: "fe94a8257192401c8cc2c45f5acc04d7"
format_stamp: "Formatted at 2026-06-19 02:39:40 on dist-test-slave-zpbg"
server_key: "28abbb1f21975e31088d6ab2ad4e1097"
server_key_iv: "03b15187909daf4c602c85bfb65f5ca6"
server_key_version: "encryptionkey@0"
I20260619 02:39:40.459177 7889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-0/wal/instance:
uuid: "fe94a8257192401c8cc2c45f5acc04d7"
format_stamp: "Formatted at 2026-06-19 02:39:40 on dist-test-slave-zpbg"
server_key: "28abbb1f21975e31088d6ab2ad4e1097"
server_key_iv: "03b15187909daf4c602c85bfb65f5ca6"
server_key_version: "encryptionkey@0"
I20260619 02:39:40.460625 7889 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:39:40.461438 7904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:40.461620 7889 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:39:40.461714 7889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-0/wal
uuid: "fe94a8257192401c8cc2c45f5acc04d7"
format_stamp: "Formatted at 2026-06-19 02:39:40 on dist-test-slave-zpbg"
server_key: "28abbb1f21975e31088d6ab2ad4e1097"
server_key_iv: "03b15187909daf4c602c85bfb65f5ca6"
server_key_version: "encryptionkey@0"
I20260619 02:39:40.461807 7889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:39:40.486951 7889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:39:40.489467 7889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:39:40.489645 7889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:39:40.490281 7889 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:39:40.490334 7889 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:40.490370 7889 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:39:40.490399 7889 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 02:39:40 dist-test-slave-zpbg krb5kdc[7772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836780, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.65@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:39:40.497989 7889 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.65:40611
I20260619 02:39:40.498065 8019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.65:40611 every 8 connection(s)
I20260619 02:39:40.498351 7889 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
I20260619 02:39:40.498713 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 7889
I20260619 02:39:40.498832 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-0/wal/instance
I20260619 02:39:40.499058 6085 external_mini_cluster.cc:1428] Setting key 028191350bbd741b22a7409887643abd
I20260619 02:39:40.501308 7867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:40.490745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:41353 (local address 127.5.241.126:46063)
0619 02:39:40.490820 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:40.490822 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:40.491134 (+ 312us) server_negotiation.cc:407] Connection header received
0619 02:39:40.491874 (+ 740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:40.491876 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:40.491888 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:40.491954 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:40.492890 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:40.493416 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:40.495121 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:40.495359 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:40.495425 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:40.498143 (+ 2718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:40.498144 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:40.498145 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:40.498153 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:40.499958 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:40.500332 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:40.500333 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:40.500333 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:40.500396 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:40.500608 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:40.500608 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:40.500609 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:40.500755 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:40.500836 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:40.501127 (+ 291us) server_negotiation.cc:299] Negotiation successful
0619 02:39:40.501219 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 02:39:40.501482 7913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:40.490725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:46063 (local address 127.5.241.65:41353)
0619 02:39:40.490956 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:40.491013 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:40.491738 (+ 725us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:40.492042 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:40.492046 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:40.492454 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:39:40.492781 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:40.492789 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:40.493525 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:40.493525 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:40.495018 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:40.495022 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:40.495445 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:40.495446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:40.495609 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:40.496144 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:39:40.496155 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:39:40.498056 (+ 1901us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:39:40.500038 (+ 1982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:40.500044 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:40.500046 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:40.500259 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:40.500459 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:40.500460 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:40.500460 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:40.500550 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:40.500823 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:39:40.500826 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:39:40.500938 (+ 112us) client_negotiation.cc:769] Sending connection context
0619 02:39:40.501087 (+ 149us) client_negotiation.cc:240] Negotiation successful
0619 02:39:40.501247 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":119,"threads_started":1}
I20260619 02:39:40.501909 8020 heartbeater.cc:347] Connected to a master server at 127.5.241.126:46063
I20260619 02:39:40.502003 8020 heartbeater.cc:464] Registering TS with master...
I20260619 02:39:40.502208 8020 heartbeater.cc:511] Master 127.5.241.126:46063 requested a full tablet report, sending...
I20260619 02:39:40.502755 7816 ts_manager.cc:194] Registered new tserver with Master: fe94a8257192401c8cc2c45f5acc04d7 (127.5.241.65:40611)
I20260619 02:39:40.503741 7816 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.65@KRBTEST.COM'} at 127.5.241.65:41353
WARNING: no policy specified for kudu/127.5.241.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.66@KRBTEST.COM" created.
I20260619 02:39:40.510439 7867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:40.504178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:55759 (local address 127.5.241.126:46063)
0619 02:39:40.504215 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:40.504217 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:40.505007 (+ 790us) server_negotiation.cc:407] Connection header received
0619 02:39:40.505079 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:40.505080 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:40.505091 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:40.505141 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:40.505485 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:40.505964 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:40.506664 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:40.506939 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:40.507022 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:40.507992 (+ 970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:40.507994 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:40.507994 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:40.508002 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:40.509369 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:40.509671 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:40.509672 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:40.509672 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:40.509721 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:40.509934 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:40.509935 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:40.509935 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:40.510086 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:40.510155 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:40.510302 (+ 147us) server_negotiation.cc:299] Negotiation successful
0619 02:39:40.510377 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 02:39:40.510540 8027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:40.504110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:46063 (local address 127.5.241.65:55759)
0619 02:39:40.504910 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:40.504921 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:40.505018 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:40.505145 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:40.505145 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:40.505334 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:39:40.505397 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:40.505399 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:40.506055 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:40.506056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:40.506567 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:40.506568 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:40.507023 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:40.507024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:40.507107 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:40.507567 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:39:40.507573 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:39:40.507886 (+ 313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:39:40.509443 (+ 1557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:40.509444 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:40.509444 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:40.509608 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:40.509784 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:40.509785 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:40.509785 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:40.509872 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:40.510144 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:39:40.510145 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:39:40.510183 (+ 38us) client_negotiation.cc:769] Sending connection context
0619 02:39:40.510250 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 02:39:40.510388 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":776,"thread_start_us":725,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab.
Entry for principal kudu/127.5.241.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab.
I20260619 02:39:40.523994 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.66:0
--local_ip_for_outbound_sockets=127.5.241.66
--webserver_interface=127.5.241.66
--webserver_port=0
--tserver_master_addrs=127.5.241.126:46063
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab
--principal=kudu/127.5.241.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:45145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:39:40.604346 8029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:39:40.604552 8029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:39:40.604575 8029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:39:40.605715 8029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:39:40.605758 8029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:39:40.605778 8029 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 02:39:40.605790 8029 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 02:39:40.605835 8029 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.66
I20260619 02:39:40.607350 8029 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:45145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab
--principal=kudu/127.5.241.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.241.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.241.126:46063
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.66
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:39:40.607585 8029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:39:40.607815 8029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:39:40.612706 8034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:40.612823 8035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:39:40.613056 8029 server_base.cc:1034] running on GCE node
W20260619 02:39:40.612706 8037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:39:40.613292 8029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:39:40.613499 8029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:39:40.614660 8029 hybrid_clock.cc:648] HybridClock initialized: now 1781836780614634 us; error 55 us; skew 500 ppm
Jun 19 02:39:40 dist-test-slave-zpbg krb5kdc[7772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836780, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:39:40.617260 8029 init.cc:377] Logged in from keytab as kudu/127.5.241.66@KRBTEST.COM (short username kudu)
I20260619 02:39:40.617686 8029 webserver.cc:460] Webserver started at http://127.5.241.66:40977/ using document root <none> and password file <none>
I20260619 02:39:40.617914 8029 fs_manager.cc:359] Metadata directory not provided
I20260619 02:39:40.617983 8029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:39:40.618145 8029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:39:40.619311 8029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-1/data/instance:
uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf"
format_stamp: "Formatted at 2026-06-19 02:39:40 on dist-test-slave-zpbg"
server_key: "9e802e70593d3935f23e3f9a7ebe6033"
server_key_iv: "b724111f45b4122c802954c12cc3b100"
server_key_version: "encryptionkey@0"
I20260619 02:39:40.619674 8029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-1/wal/instance:
uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf"
format_stamp: "Formatted at 2026-06-19 02:39:40 on dist-test-slave-zpbg"
server_key: "9e802e70593d3935f23e3f9a7ebe6033"
server_key_iv: "b724111f45b4122c802954c12cc3b100"
server_key_version: "encryptionkey@0"
I20260619 02:39:40.620986 8029 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260619 02:39:40.621764 8044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:40.621951 8029 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:39:40.622067 8029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-1/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-1/wal
uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf"
format_stamp: "Formatted at 2026-06-19 02:39:40 on dist-test-slave-zpbg"
server_key: "9e802e70593d3935f23e3f9a7ebe6033"
server_key_iv: "b724111f45b4122c802954c12cc3b100"
server_key_version: "encryptionkey@0"
I20260619 02:39:40.622152 8029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:39:40.634979 8029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:39:40.637421 8029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:39:40.637599 8029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:39:40.638121 8029 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:39:40.638187 8029 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:40.638232 8029 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:39:40.638262 8029 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:40.645431 8029 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.66:37463
I20260619 02:39:40.645515 8159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.66:37463 every 8 connection(s)
Jun 19 02:39:40 dist-test-slave-zpbg krb5kdc[7772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836780, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.66@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:39:40.645834 8029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
I20260619 02:39:40.648615 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 8029
I20260619 02:39:40.648698 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-1/wal/instance
I20260619 02:39:40.648897 6085 external_mini_cluster.cc:1428] Setting key b4aa045a7317131fd81415b054944a19
I20260619 02:39:40.649129 8053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:40.638690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:46063 (local address 127.5.241.66:38293)
0619 02:39:40.638870 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:40.638909 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:40.639447 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:40.639718 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:40.639722 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:40.640135 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:39:40.640433 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:40.640440 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:40.641478 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:40.641479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:40.643032 (+ 1553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:40.643036 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:40.643557 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:40.643558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:40.643727 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:40.644410 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:39:40.644419 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:39:40.645998 (+ 1579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:39:40.647744 (+ 1746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:40.647750 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:40.647753 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:40.647968 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:40.648151 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:40.648151 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:40.648152 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:40.648244 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:40.648524 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:39:40.648527 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:39:40.648638 (+ 111us) client_negotiation.cc:769] Sending connection context
0619 02:39:40.648766 (+ 128us) client_negotiation.cc:240] Negotiation successful
0619 02:39:40.648910 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":110,"threads_started":1}
I20260619 02:39:40.649497 7867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:40.638696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:38293 (local address 127.5.241.126:46063)
0619 02:39:40.638742 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:40.638744 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:40.639010 (+ 266us) server_negotiation.cc:407] Connection header received
0619 02:39:40.639563 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:40.639564 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:40.639576 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:40.639644 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:40.640525 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:40.641360 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:40.643131 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:40.643450 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:40.643521 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:40.646130 (+ 2609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:40.646132 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:40.646132 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:40.646140 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:40.647663 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:40.648047 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:40.648048 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:40.648048 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:40.648095 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:40.648323 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:40.648323 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:40.648324 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:40.648459 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:40.648532 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:40.649321 (+ 789us) server_negotiation.cc:299] Negotiation successful
0619 02:39:40.649413 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 02:39:40.649839 8160 heartbeater.cc:347] Connected to a master server at 127.5.241.126:46063
I20260619 02:39:40.649920 8160 heartbeater.cc:464] Registering TS with master...
I20260619 02:39:40.650064 8160 heartbeater.cc:511] Master 127.5.241.126:46063 requested a full tablet report, sending...
I20260619 02:39:40.650369 7817 ts_manager.cc:194] Registered new tserver with Master: 5b1a4ce14d2f4b6d846560a4c3e181bf (127.5.241.66:37463)
I20260619 02:39:40.650807 7817 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.66@KRBTEST.COM'} at 127.5.241.66:38293
WARNING: no policy specified for kudu/127.5.241.67@KRBTEST.COM; defaulting to no policy
I20260619 02:39:40.656910 8167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:40.651136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:46063 (local address 127.5.241.66:42839)
0619 02:39:40.651312 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:40.651323 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:40.651413 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:40.651543 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:40.651543 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:40.651732 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:39:40.651782 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:40.651783 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:40.652440 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:40.652441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:40.652959 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:40.652961 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:40.653361 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:40.653362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:40.653443 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:40.653944 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:39:40.653949 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:39:40.654235 (+ 286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:39:40.655839 (+ 1604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:40.655840 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:40.655840 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:40.656015 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:40.656224 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:40.656225 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:40.656225 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:40.656284 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:40.656573 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:39:40.656574 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:39:40.656621 (+ 47us) client_negotiation.cc:769] Sending connection context
0619 02:39:40.656708 (+ 87us) client_negotiation.cc:240] Negotiation successful
0619 02:39:40.656817 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":118,"threads_started":1}
I20260619 02:39:40.656904 7867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:40.650965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:42839 (local address 127.5.241.126:46063)
0619 02:39:40.651093 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:40.651095 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:40.651397 (+ 302us) server_negotiation.cc:407] Connection header received
0619 02:39:40.651472 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:40.651473 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:40.651483 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:40.651538 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:40.651868 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:40.652348 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:40.653059 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:40.653269 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:40.653360 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:40.654374 (+ 1014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:40.654375 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:40.654375 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:40.654382 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:40.655751 (+ 1369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:40.656101 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:40.656102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:40.656102 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:40.656158 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:40.656358 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:40.656359 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:40.656359 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:40.656500 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:40.656580 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:40.656733 (+ 153us) server_negotiation.cc:299] Negotiation successful
0619 02:39:40.656808 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab.
Entry for principal kudu/127.5.241.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab.
I20260619 02:39:40.678057 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.67:0
--local_ip_for_outbound_sockets=127.5.241.67
--webserver_interface=127.5.241.67
--webserver_port=0
--tserver_master_addrs=127.5.241.126:46063
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab
--principal=kudu/127.5.241.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:45145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:39:40.759488 8169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:39:40.759711 8169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:39:40.759737 8169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:39:40.760933 8169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:39:40.760980 8169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:39:40.761001 8169 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 02:39:40.761013 8169 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 02:39:40.761031 8169 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.67
I20260619 02:39:40.762580 8169 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:45145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab
--principal=kudu/127.5.241.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.241.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.241.126:46063
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8169
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.67
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:39:40.762830 8169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:39:40.763114 8169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:39:40.768126 8174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:40.768155 8177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:39:40.768242 8169 server_base.cc:1034] running on GCE node
W20260619 02:39:40.768167 8175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:39:40.768587 8169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:39:40.768860 8169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:39:40.770033 8169 hybrid_clock.cc:648] HybridClock initialized: now 1781836780770015 us; error 39 us; skew 500 ppm
Jun 19 02:39:40 dist-test-slave-zpbg krb5kdc[7772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836780, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:39:40.772773 8169 init.cc:377] Logged in from keytab as kudu/127.5.241.67@KRBTEST.COM (short username kudu)
I20260619 02:39:40.773279 8169 webserver.cc:460] Webserver started at http://127.5.241.67:35781/ using document root <none> and password file <none>
I20260619 02:39:40.773523 8169 fs_manager.cc:359] Metadata directory not provided
I20260619 02:39:40.773571 8169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:39:40.773717 8169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:39:40.774850 8169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-2/data/instance:
uuid: "36fdc731da6f4a7da1be563b3720ae12"
format_stamp: "Formatted at 2026-06-19 02:39:40 on dist-test-slave-zpbg"
server_key: "9a83460a3372d309eeac250c7d97baee"
server_key_iv: "f55c822fbaf0e0ac80a88eb265a37eae"
server_key_version: "encryptionkey@0"
I20260619 02:39:40.775233 8169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-2/wal/instance:
uuid: "36fdc731da6f4a7da1be563b3720ae12"
format_stamp: "Formatted at 2026-06-19 02:39:40 on dist-test-slave-zpbg"
server_key: "9a83460a3372d309eeac250c7d97baee"
server_key_iv: "f55c822fbaf0e0ac80a88eb265a37eae"
server_key_version: "encryptionkey@0"
I20260619 02:39:40.776597 8169 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:39:40.777386 8184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:40.777602 8169 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:39:40.777709 8169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-2/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-2/wal
uuid: "36fdc731da6f4a7da1be563b3720ae12"
format_stamp: "Formatted at 2026-06-19 02:39:40 on dist-test-slave-zpbg"
server_key: "9a83460a3372d309eeac250c7d97baee"
server_key_iv: "f55c822fbaf0e0ac80a88eb265a37eae"
server_key_version: "encryptionkey@0"
I20260619 02:39:40.777794 8169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:39:40.786966 8169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:39:40.788990 8169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:39:40.789145 8169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:39:40.789692 8169 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:39:40.789757 8169 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:40.789795 8169 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:39:40.789821 8169 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 02:39:40 dist-test-slave-zpbg krb5kdc[7772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836780, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.67@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:39:40.797200 8169 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.67:38815
I20260619 02:39:40.797657 8169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
I20260619 02:39:40.798125 8299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.67:38815 every 8 connection(s)
I20260619 02:39:40.800284 7867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:40.790089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:36251 (local address 127.5.241.126:46063)
0619 02:39:40.790160 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:40.790163 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:40.790392 (+ 229us) server_negotiation.cc:407] Connection header received
0619 02:39:40.790958 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:40.790960 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:40.790971 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:40.791026 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:40.791679 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:40.792189 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:40.793331 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:40.793601 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:40.793696 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:40.797080 (+ 3384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:40.797083 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:40.797084 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:40.797092 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:40.799084 (+ 1992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:40.799457 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:40.799458 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:40.799458 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:40.799507 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:40.799711 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:40.799712 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:40.799712 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:40.799811 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:40.799860 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:40.800130 (+ 270us) server_negotiation.cc:299] Negotiation successful
0619 02:39:40.800221 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 02:39:40.800333 8193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:40.790085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:46063 (local address 127.5.241.67:36251)
0619 02:39:40.790261 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:40.790299 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:40.790860 (+ 561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:40.791102 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:40.791106 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:40.791402 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:39:40.791578 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:40.791583 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:40.792280 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:40.792281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:40.793233 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:40.793235 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:40.793693 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:40.793694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:40.793817 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:40.794317 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:39:40.794327 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:39:40.796920 (+ 2593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:39:40.799156 (+ 2236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:40.799161 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:40.799162 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:40.799380 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:40.799563 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:40.799564 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:40.799564 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:40.799659 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:40.799864 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:39:40.799868 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:39:40.799982 (+ 114us) client_negotiation.cc:769] Sending connection context
0619 02:39:40.800063 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 02:39:40.800181 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":107,"threads_started":1}
I20260619 02:39:40.800815 8300 heartbeater.cc:347] Connected to a master server at 127.5.241.126:46063
I20260619 02:39:40.800930 8300 heartbeater.cc:464] Registering TS with master...
I20260619 02:39:40.801106 8300 heartbeater.cc:511] Master 127.5.241.126:46063 requested a full tablet report, sending...
I20260619 02:39:40.801481 7817 ts_manager.cc:194] Registered new tserver with Master: 36fdc731da6f4a7da1be563b3720ae12 (127.5.241.67:38815)
I20260619 02:39:40.802014 7817 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.67@KRBTEST.COM'} at 127.5.241.67:36251
I20260619 02:39:40.803165 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 8169
I20260619 02:39:40.803268 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-2/wal/instance
I20260619 02:39:40.803476 6085 external_mini_cluster.cc:1428] Setting key b0a96c201958f923c4860f2657bd90c4
I20260619 02:39:40.804179 6085 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:39:40.808233 7867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:40.801902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:43247 (local address 127.5.241.126:46063)
0619 02:39:40.801943 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:40.801945 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:40.802013 (+ 68us) server_negotiation.cc:407] Connection header received
0619 02:39:40.802081 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:40.802082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:40.802091 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:40.802148 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:40.802550 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:40.803186 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:40.804261 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:40.804486 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:40.804538 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:40.805598 (+ 1060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:40.805600 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:40.805600 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:40.805607 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:40.807177 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:40.807563 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:40.807564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:40.807564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:40.807611 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:40.807825 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:40.807825 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:40.807826 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:40.807919 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:40.807970 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:40.808123 (+ 153us) server_negotiation.cc:299] Negotiation successful
0619 02:39:40.808175 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 02:39:40.808375 8306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:40.801807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:46063 (local address 127.5.241.67:43247)
0619 02:39:40.801955 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:40.801966 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:40.802025 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:40.802202 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:40.802203 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:40.802423 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:39:40.802474 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:40.802476 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:40.803347 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:40.803348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:40.804186 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:40.804189 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:40.804555 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:40.804556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:40.804648 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:40.805075 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:39:40.805081 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:39:40.805498 (+ 417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:39:40.807255 (+ 1757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:40.807256 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:40.807256 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:40.807487 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:40.807669 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:40.807670 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:40.807671 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:40.807741 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:40.807983 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:39:40.807984 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:39:40.808032 (+ 48us) client_negotiation.cc:769] Sending connection context
0619 02:39:40.808109 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 02:39:40.808223 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":100,"threads_started":1}
Jun 19 02:39:40 dist-test-slave-zpbg krb5kdc[7772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 02:39:40 dist-test-slave-zpbg krb5kdc[7772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:39:40.827883 7867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:40.820872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56822 (local address 127.5.241.126:46063)
0619 02:39:40.820945 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:40.820947 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:40.820957 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:39:40.821024 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:40.821024 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:40.821033 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:40.821081 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:40.821616 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:40.822101 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:40.822985 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:40.823165 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:40.823213 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:40.825502 (+ 2289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:40.825503 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:40.825504 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:40.825510 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:40.827026 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:40.827327 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:40.827327 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:40.827328 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:40.827375 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:40.827522 (+ 147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:40.827522 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:40.827522 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:40.827613 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:40.827680 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:40.827763 (+ 83us) server_negotiation.cc:299] Negotiation successful
0619 02:39:40.827819 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 02:39:40.829428 7817 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56822:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:39:40.831001 7817 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:39:40.837723 8322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:40.834074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59452 (local address 127.5.241.67:38815)
0619 02:39:40.834231 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:40.834235 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:40.834376 (+ 141us) server_negotiation.cc:407] Connection header received
0619 02:39:40.834417 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:40.834419 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:40.834433 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:40.834504 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:39:40.834676 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:40.835706 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:40.836772 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:40.837276 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:40.837373 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:40.837435 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:40.837591 (+ 156us) server_negotiation.cc:299] Negotiation successful
0619 02:39:40.837627 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":85,"threads_started":1}
I20260619 02:39:40.837713 8319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:40.833592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.67:38815 (local address 127.0.0.1:59452)
0619 02:39:40.834258 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:40.834268 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:40.834319 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:40.834561 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:40.834561 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:40.834563 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:39:40.834594 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:40.834596 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:40.835783 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:40.835784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:40.836690 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:40.836693 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:40.837375 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:40.837376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:40.837460 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:40.837474 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 02:39:40.837574 (+ 100us) client_negotiation.cc:240] Negotiation successful
0619 02:39:40.837626 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"thread_start_us":61,"threads_started":1}
I20260619 02:39:40.838227 8320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:40.833735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40440 (local address 127.5.241.65:40611)
0619 02:39:40.833920 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:40.833925 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:40.833934 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:39:40.833968 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:40.833969 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:40.833982 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:40.834092 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:39:40.834328 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:40.835502 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:40.836817 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:40.837538 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:40.838022 (+ 484us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:40.838082 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:40.838108 (+ 26us) server_negotiation.cc:299] Negotiation successful
0619 02:39:40.838142 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":80,"threads_started":1}
I20260619 02:39:40.838449 8234 tablet_service.cc:1467] Processing CreateTablet for tablet b099bc5fee544b8c9366ab4a5c667262 (DEFAULT_TABLE table=test-table [id=f4736d9c3e1f468294fdcc35d22e9fee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:39:40.837975 8317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:40.833403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.66:37463 (local address 127.0.0.1:35128)
0619 02:39:40.833534 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:40.833562 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:40.833633 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:40.834262 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:40.834263 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:40.834266 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:39:40.834329 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:40.834331 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:40.835372 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:40.835373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:40.836725 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:40.836727 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:40.837691 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:40.837692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:40.837744 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:40.837747 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:39:40.837841 (+ 94us) client_negotiation.cc:240] Negotiation successful
0619 02:39:40.837894 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":36,"thread_start_us":81,"threads_started":1}
I20260619 02:39:40.838876 8234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b099bc5fee544b8c9366ab4a5c667262. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:39:40.838894 8318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:40.833399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:40611 (local address 127.0.0.1:40440)
0619 02:39:40.833533 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:40.833568 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:40.833629 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:40.834096 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:40.834099 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:40.834101 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:39:40.834173 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:40.834175 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:40.835584 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:40.835585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:40.836708 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:40.836711 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:40.837639 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:40.837640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:40.837717 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:40.837722 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 02:39:40.837906 (+ 184us) client_negotiation.cc:240] Negotiation successful
0619 02:39:40.837941 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"mutex_wait_us":2,"thread_start_us":43,"threads_started":1}
I20260619 02:39:40.839170 8321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:40.833900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35128 (local address 127.5.241.66:37463)
0619 02:39:40.834039 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:40.834044 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:40.834055 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:39:40.834094 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:40.834096 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:40.834113 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:40.834398 (+ 285us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:39:40.834457 (+ 59us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:40.835280 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:40.836814 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:40.837538 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:40.837611 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:40.837664 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:40.839031 (+ 1367us) server_negotiation.cc:299] Negotiation successful
0619 02:39:40.839074 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":80,"threads_started":1}
I20260619 02:39:40.839531 7954 tablet_service.cc:1467] Processing CreateTablet for tablet b099bc5fee544b8c9366ab4a5c667262 (DEFAULT_TABLE table=test-table [id=f4736d9c3e1f468294fdcc35d22e9fee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:39:40.839879 7954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b099bc5fee544b8c9366ab4a5c667262. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:39:40.842288 8323 tablet_bootstrap.cc:492] T b099bc5fee544b8c9366ab4a5c667262 P 36fdc731da6f4a7da1be563b3720ae12: Bootstrap starting.
I20260619 02:39:40.842887 8094 tablet_service.cc:1467] Processing CreateTablet for tablet b099bc5fee544b8c9366ab4a5c667262 (DEFAULT_TABLE table=test-table [id=f4736d9c3e1f468294fdcc35d22e9fee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:39:40.843261 8094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b099bc5fee544b8c9366ab4a5c667262. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:39:40.843405 8323 tablet_bootstrap.cc:654] T b099bc5fee544b8c9366ab4a5c667262 P 36fdc731da6f4a7da1be563b3720ae12: Neither blocks nor log segments found. Creating new log.
I20260619 02:39:40.843716 8323 log.cc:824] T b099bc5fee544b8c9366ab4a5c667262 P 36fdc731da6f4a7da1be563b3720ae12: Log is configured to *not* fsync() on all Append() calls
I20260619 02:39:40.843765 8324 tablet_bootstrap.cc:492] T b099bc5fee544b8c9366ab4a5c667262 P fe94a8257192401c8cc2c45f5acc04d7: Bootstrap starting.
I20260619 02:39:40.844360 8324 tablet_bootstrap.cc:654] T b099bc5fee544b8c9366ab4a5c667262 P fe94a8257192401c8cc2c45f5acc04d7: Neither blocks nor log segments found. Creating new log.
I20260619 02:39:40.844424 8323 tablet_bootstrap.cc:492] T b099bc5fee544b8c9366ab4a5c667262 P 36fdc731da6f4a7da1be563b3720ae12: No bootstrap required, opened a new log
I20260619 02:39:40.844491 8323 ts_tablet_manager.cc:1397] T b099bc5fee544b8c9366ab4a5c667262 P 36fdc731da6f4a7da1be563b3720ae12: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:39:40.844696 8324 log.cc:824] T b099bc5fee544b8c9366ab4a5c667262 P fe94a8257192401c8cc2c45f5acc04d7: Log is configured to *not* fsync() on all Append() calls
I20260619 02:39:40.846136 8323 raft_consensus.cc:348] T b099bc5fee544b8c9366ab4a5c667262 P 36fdc731da6f4a7da1be563b3720ae12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } }
I20260619 02:39:40.846302 8323 raft_consensus.cc:374] T b099bc5fee544b8c9366ab4a5c667262 P 36fdc731da6f4a7da1be563b3720ae12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:39:40.846343 8323 raft_consensus.cc:729] T b099bc5fee544b8c9366ab4a5c667262 P 36fdc731da6f4a7da1be563b3720ae12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36fdc731da6f4a7da1be563b3720ae12, State: Initialized, Role: FOLLOWER
I20260619 02:39:40.846410 8327 tablet_bootstrap.cc:492] T b099bc5fee544b8c9366ab4a5c667262 P 5b1a4ce14d2f4b6d846560a4c3e181bf: Bootstrap starting.
I20260619 02:39:40.846458 8323 consensus_queue.cc:260] T b099bc5fee544b8c9366ab4a5c667262 P 36fdc731da6f4a7da1be563b3720ae12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } }
I20260619 02:39:40.846714 8323 ts_tablet_manager.cc:1428] T b099bc5fee544b8c9366ab4a5c667262 P 36fdc731da6f4a7da1be563b3720ae12: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:39:40.846885 8300 heartbeater.cc:503] Master 127.5.241.126:46063 was elected leader, sending a full tablet report...
I20260619 02:39:40.847049 8327 tablet_bootstrap.cc:654] T b099bc5fee544b8c9366ab4a5c667262 P 5b1a4ce14d2f4b6d846560a4c3e181bf: Neither blocks nor log segments found. Creating new log.
I20260619 02:39:40.847074 8324 tablet_bootstrap.cc:492] T b099bc5fee544b8c9366ab4a5c667262 P fe94a8257192401c8cc2c45f5acc04d7: No bootstrap required, opened a new log
I20260619 02:39:40.847128 8324 ts_tablet_manager.cc:1397] T b099bc5fee544b8c9366ab4a5c667262 P fe94a8257192401c8cc2c45f5acc04d7: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260619 02:39:40.847375 8327 log.cc:824] T b099bc5fee544b8c9366ab4a5c667262 P 5b1a4ce14d2f4b6d846560a4c3e181bf: Log is configured to *not* fsync() on all Append() calls
I20260619 02:39:40.848177 8327 tablet_bootstrap.cc:492] T b099bc5fee544b8c9366ab4a5c667262 P 5b1a4ce14d2f4b6d846560a4c3e181bf: No bootstrap required, opened a new log
I20260619 02:39:40.848245 8327 ts_tablet_manager.cc:1397] T b099bc5fee544b8c9366ab4a5c667262 P 5b1a4ce14d2f4b6d846560a4c3e181bf: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 02:39:40.849092 8324 raft_consensus.cc:348] T b099bc5fee544b8c9366ab4a5c667262 P fe94a8257192401c8cc2c45f5acc04d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } }
I20260619 02:39:40.849260 8324 raft_consensus.cc:374] T b099bc5fee544b8c9366ab4a5c667262 P fe94a8257192401c8cc2c45f5acc04d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:39:40.849298 8324 raft_consensus.cc:729] T b099bc5fee544b8c9366ab4a5c667262 P fe94a8257192401c8cc2c45f5acc04d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe94a8257192401c8cc2c45f5acc04d7, State: Initialized, Role: FOLLOWER
I20260619 02:39:40.849408 8324 consensus_queue.cc:260] T b099bc5fee544b8c9366ab4a5c667262 P fe94a8257192401c8cc2c45f5acc04d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } }
I20260619 02:39:40.849665 8324 ts_tablet_manager.cc:1428] T b099bc5fee544b8c9366ab4a5c667262 P fe94a8257192401c8cc2c45f5acc04d7: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 02:39:40.849694 8020 heartbeater.cc:503] Master 127.5.241.126:46063 was elected leader, sending a full tablet report...
I20260619 02:39:40.850314 8327 raft_consensus.cc:348] T b099bc5fee544b8c9366ab4a5c667262 P 5b1a4ce14d2f4b6d846560a4c3e181bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } }
I20260619 02:39:40.850425 8327 raft_consensus.cc:374] T b099bc5fee544b8c9366ab4a5c667262 P 5b1a4ce14d2f4b6d846560a4c3e181bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:39:40.850487 8327 raft_consensus.cc:729] T b099bc5fee544b8c9366ab4a5c667262 P 5b1a4ce14d2f4b6d846560a4c3e181bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b1a4ce14d2f4b6d846560a4c3e181bf, State: Initialized, Role: FOLLOWER
I20260619 02:39:40.850591 8327 consensus_queue.cc:260] T b099bc5fee544b8c9366ab4a5c667262 P 5b1a4ce14d2f4b6d846560a4c3e181bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } }
I20260619 02:39:40.850857 8327 ts_tablet_manager.cc:1428] T b099bc5fee544b8c9366ab4a5c667262 P 5b1a4ce14d2f4b6d846560a4c3e181bf: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 02:39:40.850939 8160 heartbeater.cc:503] Master 127.5.241.126:46063 was elected leader, sending a full tablet report...
W20260619 02:39:40.896585 8161 tablet.cc:2369] T b099bc5fee544b8c9366ab4a5c667262 P 5b1a4ce14d2f4b6d846560a4c3e181bf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:39:40.999672 8021 tablet.cc:2369] T b099bc5fee544b8c9366ab4a5c667262 P fe94a8257192401c8cc2c45f5acc04d7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:39:41.048703 8301 tablet.cc:2369] T b099bc5fee544b8c9366ab4a5c667262 P 36fdc731da6f4a7da1be563b3720ae12: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:39:41.111538 8328 raft_consensus.cc:482] T b099bc5fee544b8c9366ab4a5c667262 P 36fdc731da6f4a7da1be563b3720ae12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:39:41.111708 8328 raft_consensus.cc:504] T b099bc5fee544b8c9366ab4a5c667262 P 36fdc731da6f4a7da1be563b3720ae12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } }
I20260619 02:39:41.112085 8328 leader_election.cc:288] T b099bc5fee544b8c9366ab4a5c667262 P 36fdc731da6f4a7da1be563b3720ae12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fe94a8257192401c8cc2c45f5acc04d7 (127.5.241.65:40611), 5b1a4ce14d2f4b6d846560a4c3e181bf (127.5.241.66:37463)
I20260619 02:39:41.114907 8193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:41.112303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:40611 (local address 127.5.241.67:47673)
0619 02:39:41.112361 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:41.112377 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:41.112447 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:41.112605 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:41.112606 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:41.112608 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:39:41.112675 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:41.112677 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:41.113345 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:41.113346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:41.114084 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:41.114085 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:41.114620 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:41.114621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:41.114696 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:41.114699 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:39:41.114760 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 02:39:41.114820 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260619 02:39:41.114977 8320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:41.112400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:47673 (local address 127.5.241.65:40611)
0619 02:39:41.112446 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:41.112448 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:41.112459 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:39:41.112527 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:41.112528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:41.112538 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:41.112585 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:39:41.112756 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:41.113242 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:41.114175 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:41.114545 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:41.114625 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:41.114634 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:41.114861 (+ 227us) server_negotiation.cc:299] Negotiation successful
0619 02:39:41.114904 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 02:39:41.115093 8321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:41.112515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:58789 (local address 127.5.241.66:37463)
0619 02:39:41.112570 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:41.112572 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:41.112639 (+ 67us) server_negotiation.cc:407] Connection header received
0619 02:39:41.112740 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:41.112742 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:41.112751 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:41.112802 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:39:41.112966 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:41.113441 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:41.114325 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:41.114697 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:41.114782 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:41.114821 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:41.114980 (+ 159us) server_negotiation.cc:299] Negotiation successful
0619 02:39:41.115016 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 02:39:41.115094 8332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:41.112303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.66:37463 (local address 127.5.241.67:58789)
0619 02:39:41.112554 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:41.112565 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:41.112664 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:41.112804 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:41.112804 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:41.112806 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:39:41.112889 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:41.112891 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:41.113526 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:41.113527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:41.114235 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:41.114237 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:41.114798 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:41.114799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:41.114889 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:41.114892 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:39:41.114963 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 02:39:41.115001 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"mutex_wait_us":73,"thread_start_us":125,"threads_started":1}
I20260619 02:39:41.115242 7974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b099bc5fee544b8c9366ab4a5c667262" candidate_uuid: "36fdc731da6f4a7da1be563b3720ae12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe94a8257192401c8cc2c45f5acc04d7" is_pre_election: true
I20260619 02:39:41.115391 7974 raft_consensus.cc:2454] T b099bc5fee544b8c9366ab4a5c667262 P fe94a8257192401c8cc2c45f5acc04d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36fdc731da6f4a7da1be563b3720ae12 in term 0.
I20260619 02:39:41.115419 8114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b099bc5fee544b8c9366ab4a5c667262" candidate_uuid: "36fdc731da6f4a7da1be563b3720ae12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" is_pre_election: true
I20260619 02:39:41.115558 8114 raft_consensus.cc:2454] T b099bc5fee544b8c9366ab4a5c667262 P 5b1a4ce14d2f4b6d846560a4c3e181bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36fdc731da6f4a7da1be563b3720ae12 in term 0.
I20260619 02:39:41.115587 8188 leader_election.cc:302] T b099bc5fee544b8c9366ab4a5c667262 P 36fdc731da6f4a7da1be563b3720ae12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36fdc731da6f4a7da1be563b3720ae12, fe94a8257192401c8cc2c45f5acc04d7; no voters:
I20260619 02:39:41.115710 8328 raft_consensus.cc:2781] T b099bc5fee544b8c9366ab4a5c667262 P 36fdc731da6f4a7da1be563b3720ae12 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:39:41.115777 8328 raft_consensus.cc:482] T b099bc5fee544b8c9366ab4a5c667262 P 36fdc731da6f4a7da1be563b3720ae12 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:39:41.115803 8328 raft_consensus.cc:3037] T b099bc5fee544b8c9366ab4a5c667262 P 36fdc731da6f4a7da1be563b3720ae12 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:39:41.116746 8328 raft_consensus.cc:504] T b099bc5fee544b8c9366ab4a5c667262 P 36fdc731da6f4a7da1be563b3720ae12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } }
I20260619 02:39:41.116891 8328 leader_election.cc:288] T b099bc5fee544b8c9366ab4a5c667262 P 36fdc731da6f4a7da1be563b3720ae12 [CANDIDATE]: Term 1 election: Requested vote from peers fe94a8257192401c8cc2c45f5acc04d7 (127.5.241.65:40611), 5b1a4ce14d2f4b6d846560a4c3e181bf (127.5.241.66:37463)
I20260619 02:39:41.117061 7974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b099bc5fee544b8c9366ab4a5c667262" candidate_uuid: "36fdc731da6f4a7da1be563b3720ae12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe94a8257192401c8cc2c45f5acc04d7"
I20260619 02:39:41.117066 8114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b099bc5fee544b8c9366ab4a5c667262" candidate_uuid: "36fdc731da6f4a7da1be563b3720ae12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf"
I20260619 02:39:41.117147 8114 raft_consensus.cc:3037] T b099bc5fee544b8c9366ab4a5c667262 P 5b1a4ce14d2f4b6d846560a4c3e181bf [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:39:41.117147 7974 raft_consensus.cc:3037] T b099bc5fee544b8c9366ab4a5c667262 P fe94a8257192401c8cc2c45f5acc04d7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:39:41.117996 7974 raft_consensus.cc:2454] T b099bc5fee544b8c9366ab4a5c667262 P fe94a8257192401c8cc2c45f5acc04d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36fdc731da6f4a7da1be563b3720ae12 in term 1.
I20260619 02:39:41.117996 8114 raft_consensus.cc:2454] T b099bc5fee544b8c9366ab4a5c667262 P 5b1a4ce14d2f4b6d846560a4c3e181bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36fdc731da6f4a7da1be563b3720ae12 in term 1.
I20260619 02:39:41.118232 8188 leader_election.cc:302] T b099bc5fee544b8c9366ab4a5c667262 P 36fdc731da6f4a7da1be563b3720ae12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36fdc731da6f4a7da1be563b3720ae12, fe94a8257192401c8cc2c45f5acc04d7; no voters:
I20260619 02:39:41.118355 8328 raft_consensus.cc:2781] T b099bc5fee544b8c9366ab4a5c667262 P 36fdc731da6f4a7da1be563b3720ae12 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:39:41.118517 8328 raft_consensus.cc:686] T b099bc5fee544b8c9366ab4a5c667262 P 36fdc731da6f4a7da1be563b3720ae12 [term 1 LEADER]: Becoming Leader. State: Replica: 36fdc731da6f4a7da1be563b3720ae12, State: Running, Role: LEADER
I20260619 02:39:41.118618 8328 consensus_queue.cc:237] T b099bc5fee544b8c9366ab4a5c667262 P 36fdc731da6f4a7da1be563b3720ae12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } }
I20260619 02:39:41.119589 7817 catalog_manager.cc:5495] T b099bc5fee544b8c9366ab4a5c667262 P 36fdc731da6f4a7da1be563b3720ae12 reported cstate change: term changed from 0 to 1, leader changed from <none> to 36fdc731da6f4a7da1be563b3720ae12 (127.5.241.67). New cstate: current_term: 1 leader_uuid: "36fdc731da6f4a7da1be563b3720ae12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } health_report { overall_health: HEALTHY } } }
Jun 19 02:39:41 dist-test-slave-zpbg krb5kdc[7772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836780, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:39:41.214321 7867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:41.207553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56830 (local address 127.5.241.126:46063)
0619 02:39:41.207631 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:41.207633 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:41.207737 (+ 104us) server_negotiation.cc:407] Connection header received
0619 02:39:41.207831 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:41.207832 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:41.207843 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:41.207924 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:41.208390 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:41.208870 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:41.209593 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:41.209774 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:41.209859 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:41.211923 (+ 2064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:41.211924 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:41.211925 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:41.211931 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:41.213127 (+ 1196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:41.213510 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:41.213511 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:41.213512 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:41.213575 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:41.213766 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:41.213767 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:41.213767 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:41.213878 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:41.213960 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:41.214108 (+ 148us) server_negotiation.cc:299] Negotiation successful
0619 02:39:41.214209 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 02:39:41.214393 8340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:41.207449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:46063 (local address 127.0.0.1:56830)
0619 02:39:41.207655 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:41.207673 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:41.207757 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:41.207938 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:41.207940 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:41.208250 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:39:41.208293 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:41.208295 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:41.208958 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:41.208959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:41.209494 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:41.209495 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:41.209857 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:41.209858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:41.209924 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:41.210445 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:39:41.210449 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:39:41.211820 (+ 1371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:39:41.213220 (+ 1400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:41.213227 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:41.213230 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:41.213418 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:41.213642 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:41.213643 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:41.213643 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:41.213704 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:41.213949 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:39:41.213955 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:39:41.213999 (+ 44us) client_negotiation.cc:769] Sending connection context
0619 02:39:41.214085 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 02:39:41.214269 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":86,"thread_start_us":115,"threads_started":1}
I20260619 02:39:41.216503 7817 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:56830:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 02:39:41.216773 7817 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:39:41.218467 8094 tablet_service.cc:1467] Processing CreateTablet for tablet 37cab6593ec547ecb7deb9dc46a3b099 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=73b4db18ed2344a09d8bcddcfb9edff3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 02:39:41.218492 8234 tablet_service.cc:1467] Processing CreateTablet for tablet 37cab6593ec547ecb7deb9dc46a3b099 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=73b4db18ed2344a09d8bcddcfb9edff3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 02:39:41.218475 7954 tablet_service.cc:1467] Processing CreateTablet for tablet 37cab6593ec547ecb7deb9dc46a3b099 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=73b4db18ed2344a09d8bcddcfb9edff3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 02:39:41.218642 7954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37cab6593ec547ecb7deb9dc46a3b099. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:39:41.218642 8234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37cab6593ec547ecb7deb9dc46a3b099. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:39:41.218653 8094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37cab6593ec547ecb7deb9dc46a3b099. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:39:41.219933 8327 tablet_bootstrap.cc:492] T 37cab6593ec547ecb7deb9dc46a3b099 P 5b1a4ce14d2f4b6d846560a4c3e181bf: Bootstrap starting.
I20260619 02:39:41.220181 8324 tablet_bootstrap.cc:492] T 37cab6593ec547ecb7deb9dc46a3b099 P fe94a8257192401c8cc2c45f5acc04d7: Bootstrap starting.
I20260619 02:39:41.220206 8323 tablet_bootstrap.cc:492] T 37cab6593ec547ecb7deb9dc46a3b099 P 36fdc731da6f4a7da1be563b3720ae12: Bootstrap starting.
I20260619 02:39:41.220356 8327 tablet_bootstrap.cc:654] T 37cab6593ec547ecb7deb9dc46a3b099 P 5b1a4ce14d2f4b6d846560a4c3e181bf: Neither blocks nor log segments found. Creating new log.
I20260619 02:39:41.220577 8324 tablet_bootstrap.cc:654] T 37cab6593ec547ecb7deb9dc46a3b099 P fe94a8257192401c8cc2c45f5acc04d7: Neither blocks nor log segments found. Creating new log.
I20260619 02:39:41.220602 8323 tablet_bootstrap.cc:654] T 37cab6593ec547ecb7deb9dc46a3b099 P 36fdc731da6f4a7da1be563b3720ae12: Neither blocks nor log segments found. Creating new log.
I20260619 02:39:41.220966 8327 tablet_bootstrap.cc:492] T 37cab6593ec547ecb7deb9dc46a3b099 P 5b1a4ce14d2f4b6d846560a4c3e181bf: No bootstrap required, opened a new log
I20260619 02:39:41.221020 8327 ts_tablet_manager.cc:1397] T 37cab6593ec547ecb7deb9dc46a3b099 P 5b1a4ce14d2f4b6d846560a4c3e181bf: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:39:41.221201 8324 tablet_bootstrap.cc:492] T 37cab6593ec547ecb7deb9dc46a3b099 P fe94a8257192401c8cc2c45f5acc04d7: No bootstrap required, opened a new log
I20260619 02:39:41.221174 8327 raft_consensus.cc:348] T 37cab6593ec547ecb7deb9dc46a3b099 P 5b1a4ce14d2f4b6d846560a4c3e181bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } }
I20260619 02:39:41.221248 8324 ts_tablet_manager.cc:1397] T 37cab6593ec547ecb7deb9dc46a3b099 P fe94a8257192401c8cc2c45f5acc04d7: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:39:41.221254 8327 raft_consensus.cc:374] T 37cab6593ec547ecb7deb9dc46a3b099 P 5b1a4ce14d2f4b6d846560a4c3e181bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:39:41.221280 8327 raft_consensus.cc:729] T 37cab6593ec547ecb7deb9dc46a3b099 P 5b1a4ce14d2f4b6d846560a4c3e181bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b1a4ce14d2f4b6d846560a4c3e181bf, State: Initialized, Role: FOLLOWER
I20260619 02:39:41.221329 8327 consensus_queue.cc:260] T 37cab6593ec547ecb7deb9dc46a3b099 P 5b1a4ce14d2f4b6d846560a4c3e181bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } }
I20260619 02:39:41.221366 8323 tablet_bootstrap.cc:492] T 37cab6593ec547ecb7deb9dc46a3b099 P 36fdc731da6f4a7da1be563b3720ae12: No bootstrap required, opened a new log
I20260619 02:39:41.221401 8323 ts_tablet_manager.cc:1397] T 37cab6593ec547ecb7deb9dc46a3b099 P 36fdc731da6f4a7da1be563b3720ae12: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:39:41.221431 8327 ts_tablet_manager.cc:1428] T 37cab6593ec547ecb7deb9dc46a3b099 P 5b1a4ce14d2f4b6d846560a4c3e181bf: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:41.221401 8324 raft_consensus.cc:348] T 37cab6593ec547ecb7deb9dc46a3b099 P fe94a8257192401c8cc2c45f5acc04d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } }
I20260619 02:39:41.221472 8324 raft_consensus.cc:374] T 37cab6593ec547ecb7deb9dc46a3b099 P fe94a8257192401c8cc2c45f5acc04d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:39:41.221496 8324 raft_consensus.cc:729] T 37cab6593ec547ecb7deb9dc46a3b099 P fe94a8257192401c8cc2c45f5acc04d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe94a8257192401c8cc2c45f5acc04d7, State: Initialized, Role: FOLLOWER
I20260619 02:39:41.221539 8324 consensus_queue.cc:260] T 37cab6593ec547ecb7deb9dc46a3b099 P fe94a8257192401c8cc2c45f5acc04d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } }
I20260619 02:39:41.221539 8323 raft_consensus.cc:348] T 37cab6593ec547ecb7deb9dc46a3b099 P 36fdc731da6f4a7da1be563b3720ae12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } }
I20260619 02:39:41.221619 8323 raft_consensus.cc:374] T 37cab6593ec547ecb7deb9dc46a3b099 P 36fdc731da6f4a7da1be563b3720ae12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:39:41.221639 8324 ts_tablet_manager.cc:1428] T 37cab6593ec547ecb7deb9dc46a3b099 P fe94a8257192401c8cc2c45f5acc04d7: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 02:39:41.221660 8323 raft_consensus.cc:729] T 37cab6593ec547ecb7deb9dc46a3b099 P 36fdc731da6f4a7da1be563b3720ae12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36fdc731da6f4a7da1be563b3720ae12, State: Initialized, Role: FOLLOWER
I20260619 02:39:41.221701 8323 consensus_queue.cc:260] T 37cab6593ec547ecb7deb9dc46a3b099 P 36fdc731da6f4a7da1be563b3720ae12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } }
I20260619 02:39:41.221698 8330 tablet_replica.cc:439] T 37cab6593ec547ecb7deb9dc46a3b099 P fe94a8257192401c8cc2c45f5acc04d7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } } }
I20260619 02:39:41.221781 8323 ts_tablet_manager.cc:1428] T 37cab6593ec547ecb7deb9dc46a3b099 P 36fdc731da6f4a7da1be563b3720ae12: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:41.221786 8330 tablet_replica.cc:442] T 37cab6593ec547ecb7deb9dc46a3b099 P fe94a8257192401c8cc2c45f5acc04d7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:39:41.221760 8331 tablet_replica.cc:439] T 37cab6593ec547ecb7deb9dc46a3b099 P 5b1a4ce14d2f4b6d846560a4c3e181bf: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } } }
I20260619 02:39:41.221855 8331 tablet_replica.cc:442] T 37cab6593ec547ecb7deb9dc46a3b099 P 5b1a4ce14d2f4b6d846560a4c3e181bf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:39:41.221895 8328 tablet_replica.cc:439] T 37cab6593ec547ecb7deb9dc46a3b099 P 36fdc731da6f4a7da1be563b3720ae12: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } } }
I20260619 02:39:41.221969 8328 tablet_replica.cc:442] T 37cab6593ec547ecb7deb9dc46a3b099 P 36fdc731da6f4a7da1be563b3720ae12: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:39:41.405287 8331 raft_consensus.cc:482] T 37cab6593ec547ecb7deb9dc46a3b099 P 5b1a4ce14d2f4b6d846560a4c3e181bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:39:41.405390 8331 raft_consensus.cc:504] T 37cab6593ec547ecb7deb9dc46a3b099 P 5b1a4ce14d2f4b6d846560a4c3e181bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } }
I20260619 02:39:41.405776 8331 leader_election.cc:288] T 37cab6593ec547ecb7deb9dc46a3b099 P 5b1a4ce14d2f4b6d846560a4c3e181bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fe94a8257192401c8cc2c45f5acc04d7 (127.5.241.65:40611), 36fdc731da6f4a7da1be563b3720ae12 (127.5.241.67:38815)
I20260619 02:39:41.408731 8345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:41.405951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:40611 (local address 127.5.241.66:52191)
0619 02:39:41.406195 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:41.406205 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:41.406270 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:41.406418 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:41.406419 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:41.406421 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:39:41.406515 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:41.406516 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:41.407195 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:41.407196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:41.407954 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:41.407956 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:41.408507 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:41.408508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:41.408583 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:41.408588 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 02:39:41.408638 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 02:39:41.408667 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":59,"threads_started":1}
I20260619 02:39:41.408779 8320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:41.406005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:52191 (local address 127.5.241.65:40611)
0619 02:39:41.406052 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:41.406054 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:41.406294 (+ 240us) server_negotiation.cc:407] Connection header received
0619 02:39:41.406329 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:41.406330 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:41.406339 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:41.406396 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:39:41.406589 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:41.407108 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:41.408047 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:41.408410 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:41.408471 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:41.408478 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:41.408667 (+ 189us) server_negotiation.cc:299] Negotiation successful
0619 02:39:41.408703 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 02:39:41.409039 7974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37cab6593ec547ecb7deb9dc46a3b099" candidate_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe94a8257192401c8cc2c45f5acc04d7" is_pre_election: true
I20260619 02:39:41.409163 7974 raft_consensus.cc:2454] T 37cab6593ec547ecb7deb9dc46a3b099 P fe94a8257192401c8cc2c45f5acc04d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b1a4ce14d2f4b6d846560a4c3e181bf in term 0.
I20260619 02:39:41.409305 8346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:41.406030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:35231 (local address 127.5.241.67:38815)
0619 02:39:41.406168 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:41.406171 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:41.406181 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:39:41.406212 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:41.406213 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:41.406226 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:41.406346 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:39:41.406512 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:41.407063 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:41.408094 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:41.408490 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:41.408563 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:41.408573 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:41.409188 (+ 615us) server_negotiation.cc:299] Negotiation successful
0619 02:39:41.409231 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79,"thread_start_us":50,"threads_started":1}
I20260619 02:39:41.409322 8344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:41.405944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.67:38815 (local address 127.5.241.66:35231)
0619 02:39:41.406073 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:41.406085 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:41.406140 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:41.406358 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:41.406359 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:41.406362 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:39:41.406431 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:41.406433 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:41.407143 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:41.407144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:41.407978 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:41.407980 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:41.408575 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:41.408575 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:41.409102 (+ 527us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:41.409105 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:39:41.409181 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 02:39:41.409220 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":44,"thread_start_us":56,"threads_started":1}
I20260619 02:39:41.409339 8048 leader_election.cc:302] T 37cab6593ec547ecb7deb9dc46a3b099 P 5b1a4ce14d2f4b6d846560a4c3e181bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b1a4ce14d2f4b6d846560a4c3e181bf, fe94a8257192401c8cc2c45f5acc04d7; no voters:
I20260619 02:39:41.409480 8331 raft_consensus.cc:2781] T 37cab6593ec547ecb7deb9dc46a3b099 P 5b1a4ce14d2f4b6d846560a4c3e181bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:39:41.409528 8331 raft_consensus.cc:482] T 37cab6593ec547ecb7deb9dc46a3b099 P 5b1a4ce14d2f4b6d846560a4c3e181bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:39:41.409550 8331 raft_consensus.cc:3037] T 37cab6593ec547ecb7deb9dc46a3b099 P 5b1a4ce14d2f4b6d846560a4c3e181bf [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:39:41.409663 8254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37cab6593ec547ecb7deb9dc46a3b099" candidate_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36fdc731da6f4a7da1be563b3720ae12" is_pre_election: true
I20260619 02:39:41.409777 8254 raft_consensus.cc:2454] T 37cab6593ec547ecb7deb9dc46a3b099 P 36fdc731da6f4a7da1be563b3720ae12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b1a4ce14d2f4b6d846560a4c3e181bf in term 0.
I20260619 02:39:41.410344 8331 raft_consensus.cc:504] T 37cab6593ec547ecb7deb9dc46a3b099 P 5b1a4ce14d2f4b6d846560a4c3e181bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } }
I20260619 02:39:41.410526 8331 leader_election.cc:288] T 37cab6593ec547ecb7deb9dc46a3b099 P 5b1a4ce14d2f4b6d846560a4c3e181bf [CANDIDATE]: Term 1 election: Requested vote from peers fe94a8257192401c8cc2c45f5acc04d7 (127.5.241.65:40611), 36fdc731da6f4a7da1be563b3720ae12 (127.5.241.67:38815)
I20260619 02:39:41.410655 7974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37cab6593ec547ecb7deb9dc46a3b099" candidate_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe94a8257192401c8cc2c45f5acc04d7"
I20260619 02:39:41.410667 8254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37cab6593ec547ecb7deb9dc46a3b099" candidate_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36fdc731da6f4a7da1be563b3720ae12"
I20260619 02:39:41.410733 7974 raft_consensus.cc:3037] T 37cab6593ec547ecb7deb9dc46a3b099 P fe94a8257192401c8cc2c45f5acc04d7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:39:41.410753 8254 raft_consensus.cc:3037] T 37cab6593ec547ecb7deb9dc46a3b099 P 36fdc731da6f4a7da1be563b3720ae12 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:39:41.411342 8254 raft_consensus.cc:2454] T 37cab6593ec547ecb7deb9dc46a3b099 P 36fdc731da6f4a7da1be563b3720ae12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b1a4ce14d2f4b6d846560a4c3e181bf in term 1.
I20260619 02:39:41.411474 7974 raft_consensus.cc:2454] T 37cab6593ec547ecb7deb9dc46a3b099 P fe94a8257192401c8cc2c45f5acc04d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b1a4ce14d2f4b6d846560a4c3e181bf in term 1.
I20260619 02:39:41.411573 8045 leader_election.cc:302] T 37cab6593ec547ecb7deb9dc46a3b099 P 5b1a4ce14d2f4b6d846560a4c3e181bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36fdc731da6f4a7da1be563b3720ae12, 5b1a4ce14d2f4b6d846560a4c3e181bf; no voters:
I20260619 02:39:41.411655 8331 raft_consensus.cc:2781] T 37cab6593ec547ecb7deb9dc46a3b099 P 5b1a4ce14d2f4b6d846560a4c3e181bf [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:39:41.411758 8331 raft_consensus.cc:686] T 37cab6593ec547ecb7deb9dc46a3b099 P 5b1a4ce14d2f4b6d846560a4c3e181bf [term 1 LEADER]: Becoming Leader. State: Replica: 5b1a4ce14d2f4b6d846560a4c3e181bf, State: Running, Role: LEADER
I20260619 02:39:41.411862 8331 consensus_queue.cc:237] T 37cab6593ec547ecb7deb9dc46a3b099 P 5b1a4ce14d2f4b6d846560a4c3e181bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } }
I20260619 02:39:41.412307 8347 tablet_replica.cc:439] T 37cab6593ec547ecb7deb9dc46a3b099 P 5b1a4ce14d2f4b6d846560a4c3e181bf: TxnStatusTablet state changed. Reason: New leader 5b1a4ce14d2f4b6d846560a4c3e181bf. Latest consensus state: current_term: 1 leader_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } } }
I20260619 02:39:41.412484 8347 tablet_replica.cc:442] T 37cab6593ec547ecb7deb9dc46a3b099 P 5b1a4ce14d2f4b6d846560a4c3e181bf: This TxnStatusTablet replica's current role is: LEADER
I20260619 02:39:41.412742 8349 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 02:39:41.412714 7815 catalog_manager.cc:5495] T 37cab6593ec547ecb7deb9dc46a3b099 P 5b1a4ce14d2f4b6d846560a4c3e181bf reported cstate change: term changed from 0 to 1, leader changed from <none> to 5b1a4ce14d2f4b6d846560a4c3e181bf (127.5.241.66). New cstate: current_term: 1 leader_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } health_report { overall_health: UNKNOWN } } }
I20260619 02:39:41.412844 8349 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 02:39:41.452140 8321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:41.449466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35140 (local address 127.5.241.66:37463)
0619 02:39:41.449516 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:41.449518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:41.449528 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:39:41.449626 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:41.449627 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:41.449646 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:41.449737 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:39:41.449847 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:41.450440 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:41.451300 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:41.451493 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:41.451578 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:41.451772 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:39:41.451880 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:39:41.451973 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:41.452014 (+ 41us) server_negotiation.cc:299] Negotiation successful
0619 02:39:41.452037 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 02:39:41.452150 8340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:41.449394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.66:37463 (local address 127.0.0.1:35140)
0619 02:39:41.449476 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:41.449490 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:41.449543 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:41.449734 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:41.449735 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:41.449737 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:39:41.449792 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:41.449794 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:41.450525 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:41.450526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:41.451182 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:41.451184 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:41.451582 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:41.451583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:41.451654 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:41.451669 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:39:41.451948 (+ 279us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:39:41.451957 (+ 9us) client_negotiation.cc:769] Sending connection context
0619 02:39:41.452008 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 02:39:41.452058 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260619 02:39:41.454859 7974 raft_consensus.cc:1261] T 37cab6593ec547ecb7deb9dc46a3b099 P fe94a8257192401c8cc2c45f5acc04d7 [term 1 FOLLOWER]: Refusing update from remote peer 5b1a4ce14d2f4b6d846560a4c3e181bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:39:41.454900 8254 raft_consensus.cc:1261] T 37cab6593ec547ecb7deb9dc46a3b099 P 36fdc731da6f4a7da1be563b3720ae12 [term 1 FOLLOWER]: Refusing update from remote peer 5b1a4ce14d2f4b6d846560a4c3e181bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:39:41.455190 8347 consensus_queue.cc:1035] T 37cab6593ec547ecb7deb9dc46a3b099 P 5b1a4ce14d2f4b6d846560a4c3e181bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:39:41.455333 8331 consensus_queue.cc:1035] T 37cab6593ec547ecb7deb9dc46a3b099 P 5b1a4ce14d2f4b6d846560a4c3e181bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:39:41.456545 8330 tablet_replica.cc:439] T 37cab6593ec547ecb7deb9dc46a3b099 P fe94a8257192401c8cc2c45f5acc04d7: TxnStatusTablet state changed. Reason: New leader 5b1a4ce14d2f4b6d846560a4c3e181bf. Latest consensus state: current_term: 1 leader_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } } }
I20260619 02:39:41.456638 8330 tablet_replica.cc:442] T 37cab6593ec547ecb7deb9dc46a3b099 P fe94a8257192401c8cc2c45f5acc04d7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:39:41.456696 8328 tablet_replica.cc:439] T 37cab6593ec547ecb7deb9dc46a3b099 P 36fdc731da6f4a7da1be563b3720ae12: TxnStatusTablet state changed. Reason: New leader 5b1a4ce14d2f4b6d846560a4c3e181bf. Latest consensus state: current_term: 1 leader_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } } }
I20260619 02:39:41.456768 8328 tablet_replica.cc:442] T 37cab6593ec547ecb7deb9dc46a3b099 P 36fdc731da6f4a7da1be563b3720ae12: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:39:41.457707 8331 tablet_replica.cc:439] T 37cab6593ec547ecb7deb9dc46a3b099 P 5b1a4ce14d2f4b6d846560a4c3e181bf: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } } }
I20260619 02:39:41.457790 8331 tablet_replica.cc:442] T 37cab6593ec547ecb7deb9dc46a3b099 P 5b1a4ce14d2f4b6d846560a4c3e181bf: This TxnStatusTablet replica's current role is: LEADER
I20260619 02:39:41.457836 8331 tablet_replica.cc:439] T 37cab6593ec547ecb7deb9dc46a3b099 P 5b1a4ce14d2f4b6d846560a4c3e181bf: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } } }
I20260619 02:39:41.457872 8331 tablet_replica.cc:442] T 37cab6593ec547ecb7deb9dc46a3b099 P 5b1a4ce14d2f4b6d846560a4c3e181bf: This TxnStatusTablet replica's current role is: LEADER
I20260619 02:39:41.458092 8328 tablet_replica.cc:439] T 37cab6593ec547ecb7deb9dc46a3b099 P 36fdc731da6f4a7da1be563b3720ae12: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } } }
I20260619 02:39:41.458160 8328 tablet_replica.cc:442] T 37cab6593ec547ecb7deb9dc46a3b099 P 36fdc731da6f4a7da1be563b3720ae12: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:39:41.458982 8330 tablet_replica.cc:439] T 37cab6593ec547ecb7deb9dc46a3b099 P fe94a8257192401c8cc2c45f5acc04d7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } } }
I20260619 02:39:41.459075 8330 tablet_replica.cc:442] T 37cab6593ec547ecb7deb9dc46a3b099 P fe94a8257192401c8cc2c45f5acc04d7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:39:41.459816 8354 mvcc.cc:205] Tried to move back new op lower bound from 7298403456836243456 to 7298403456663519232. Current Snapshot: MvccSnapshot[applied={T|T < 7298403456836243456 or (T in {7298403456836243456})}]
I20260619 02:39:41.462422 8346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:41.459678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59466 (local address 127.5.241.67:38815)
0619 02:39:41.459713 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:41.459715 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:41.459724 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:39:41.459778 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:41.459779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:41.459797 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:41.459844 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:39:41.460065 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:41.460854 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:41.461609 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:41.461922 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:41.461992 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:41.462112 (+ 120us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:39:41.462221 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:39:41.462295 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:41.462333 (+ 38us) server_negotiation.cc:299] Negotiation successful
0619 02:39:41.462353 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 02:39:41.467381 8321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:41.464892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:54001 (local address 127.5.241.66:37463)
0619 02:39:41.464938 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:41.464940 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:41.465059 (+ 119us) server_negotiation.cc:407] Connection header received
0619 02:39:41.465124 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:41.465125 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:41.465136 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:41.465189 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:39:41.465331 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:41.465864 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:41.466626 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:41.466899 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:41.466973 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:41.467200 (+ 227us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:39:41.467225 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:39:41.467263 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:41.467314 (+ 51us) server_negotiation.cc:299] Negotiation successful
0619 02:39:41.467330 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 02:39:41.467439 8365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:41.464841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.66:37463 (local address 127.5.241.67:54001)
0619 02:39:41.464986 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:41.464997 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:41.465075 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:41.465193 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:41.465193 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:41.465195 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:39:41.465252 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:41.465253 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:41.465961 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:41.465963 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:41.466539 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:41.466541 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:41.467016 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:41.467018 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:41.467090 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:41.467104 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:39:41.467264 (+ 160us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:39:41.467265 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 02:39:41.467312 (+ 47us) client_negotiation.cc:240] Negotiation successful
0619 02:39:41.467351 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":85,"threads_started":1}
I20260619 02:39:41.469542 7974 raft_consensus.cc:1261] T b099bc5fee544b8c9366ab4a5c667262 P fe94a8257192401c8cc2c45f5acc04d7 [term 1 FOLLOWER]: Refusing update from remote peer 36fdc731da6f4a7da1be563b3720ae12: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:39:41.469568 8114 raft_consensus.cc:1261] T b099bc5fee544b8c9366ab4a5c667262 P 5b1a4ce14d2f4b6d846560a4c3e181bf [term 1 FOLLOWER]: Refusing update from remote peer 36fdc731da6f4a7da1be563b3720ae12: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:39:41.469786 8328 consensus_queue.cc:1035] T b099bc5fee544b8c9366ab4a5c667262 P 36fdc731da6f4a7da1be563b3720ae12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:39:41.469936 8333 consensus_queue.cc:1035] T b099bc5fee544b8c9366ab4a5c667262 P 36fdc731da6f4a7da1be563b3720ae12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:39:41.471541 8352 mvcc.cc:205] Tried to move back new op lower bound from 7298403456897572864 to 7298403455462551552. Current Snapshot: MvccSnapshot[applied={T|T < 7298403456897572864}]
I20260619 02:39:41.479228 8346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:41.476064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:38777 (local address 127.5.241.67:38815)
0619 02:39:41.476108 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:41.476111 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:41.476284 (+ 173us) server_negotiation.cc:407] Connection header received
0619 02:39:41.476391 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:41.476392 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:41.476402 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:41.476431 (+ 29us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:39:41.476618 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:41.477323 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:41.478403 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:41.478693 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:41.478751 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:41.478992 (+ 241us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:39:41.479026 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:39:41.479074 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:41.479131 (+ 57us) server_negotiation.cc:299] Negotiation successful
0619 02:39:41.479153 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 02:39:41.479241 8369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:41.476051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.67:38815 (local address 127.5.241.66:38777)
0619 02:39:41.476219 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:41.476232 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:41.476311 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:41.476493 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:41.476494 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:41.476496 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:39:41.476541 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:41.476543 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:41.477425 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:41.477426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:41.478322 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:41.478326 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:41.478805 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:41.478806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:41.478865 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:41.478880 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:39:41.479073 (+ 193us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:39:41.479075 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 02:39:41.479127 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 02:39:41.479171 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":102,"threads_started":1}
I20260619 02:39:41.486434 7817 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56822:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:39:41.486550 7817 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56822:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:39:41.487468 7817 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 61c08775450f47beb0d8ab94784cf7ab: Sending DeleteTablet for 3 replicas of tablet b099bc5fee544b8c9366ab4a5c667262
I20260619 02:39:41.487818 7954 tablet_service.cc:1514] Processing DeleteTablet for tablet b099bc5fee544b8c9366ab4a5c667262 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:39:41 UTC) from {username='kudu', principal='kudu/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:40440
I20260619 02:39:41.487967 8234 tablet_service.cc:1514] Processing DeleteTablet for tablet b099bc5fee544b8c9366ab4a5c667262 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:39:41 UTC) from {username='kudu', principal='kudu/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:59452
I20260619 02:39:41.488207 8373 tablet_replica.cc:330] T b099bc5fee544b8c9366ab4a5c667262 P 36fdc731da6f4a7da1be563b3720ae12: stopping tablet replica
I20260619 02:39:41.488355 8373 raft_consensus.cc:2229] T b099bc5fee544b8c9366ab4a5c667262 P 36fdc731da6f4a7da1be563b3720ae12 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:39:41.488509 8373 raft_consensus.cc:2258] T b099bc5fee544b8c9366ab4a5c667262 P 36fdc731da6f4a7da1be563b3720ae12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:39:41.488497 8094 tablet_service.cc:1514] Processing DeleteTablet for tablet b099bc5fee544b8c9366ab4a5c667262 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:39:41 UTC) from {username='kudu', principal='kudu/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:35128
I20260619 02:39:41.488705 7817 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56822:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:39:41.488848 7817 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:39:41.488981 8373 ts_tablet_manager.cc:1905] T b099bc5fee544b8c9366ab4a5c667262 P 36fdc731da6f4a7da1be563b3720ae12: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:39:41.490280 7954 tablet_service.cc:1467] Processing CreateTablet for tablet 07f1cb156e59479eaa1580d7bb2f6e56 (DEFAULT_TABLE table=test-table [id=bdad03b0b214411dbd462d00b1c2000a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:39:41.490408 7954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07f1cb156e59479eaa1580d7bb2f6e56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:39:41.490687 8373 ts_tablet_manager.cc:1918] T b099bc5fee544b8c9366ab4a5c667262 P 36fdc731da6f4a7da1be563b3720ae12: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:39:41.490748 8373 log.cc:1196] T b099bc5fee544b8c9366ab4a5c667262 P 36fdc731da6f4a7da1be563b3720ae12: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-2/wal/wals/b099bc5fee544b8c9366ab4a5c667262
I20260619 02:39:41.491041 8373 ts_tablet_manager.cc:1939] T b099bc5fee544b8c9366ab4a5c667262 P 36fdc731da6f4a7da1be563b3720ae12: Deleting consensus metadata
I20260619 02:39:41.491871 8324 tablet_bootstrap.cc:492] T 07f1cb156e59479eaa1580d7bb2f6e56 P fe94a8257192401c8cc2c45f5acc04d7: Bootstrap starting.
I20260619 02:39:41.492240 7801 catalog_manager.cc:4841] TS 36fdc731da6f4a7da1be563b3720ae12 (127.5.241.67:38815): tablet b099bc5fee544b8c9366ab4a5c667262 (table test-table [id=f4736d9c3e1f468294fdcc35d22e9fee]) successfully deleted
I20260619 02:39:41.492537 8234 tablet_service.cc:1467] Processing CreateTablet for tablet 07f1cb156e59479eaa1580d7bb2f6e56 (DEFAULT_TABLE table=test-table [id=bdad03b0b214411dbd462d00b1c2000a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:39:41.492589 8324 tablet_bootstrap.cc:654] T 07f1cb156e59479eaa1580d7bb2f6e56 P fe94a8257192401c8cc2c45f5acc04d7: Neither blocks nor log segments found. Creating new log.
I20260619 02:39:41.492645 8234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07f1cb156e59479eaa1580d7bb2f6e56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:39:41.493209 8324 tablet_bootstrap.cc:492] T 07f1cb156e59479eaa1580d7bb2f6e56 P fe94a8257192401c8cc2c45f5acc04d7: No bootstrap required, opened a new log
I20260619 02:39:41.493269 8324 ts_tablet_manager.cc:1397] T 07f1cb156e59479eaa1580d7bb2f6e56 P fe94a8257192401c8cc2c45f5acc04d7: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:39:41.493404 8324 raft_consensus.cc:348] T 07f1cb156e59479eaa1580d7bb2f6e56 P fe94a8257192401c8cc2c45f5acc04d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } }
I20260619 02:39:41.493471 8324 raft_consensus.cc:374] T 07f1cb156e59479eaa1580d7bb2f6e56 P fe94a8257192401c8cc2c45f5acc04d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:39:41.493491 8324 raft_consensus.cc:729] T 07f1cb156e59479eaa1580d7bb2f6e56 P fe94a8257192401c8cc2c45f5acc04d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe94a8257192401c8cc2c45f5acc04d7, State: Initialized, Role: FOLLOWER
I20260619 02:39:41.493536 8324 consensus_queue.cc:260] T 07f1cb156e59479eaa1580d7bb2f6e56 P fe94a8257192401c8cc2c45f5acc04d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } }
I20260619 02:39:41.493625 8324 ts_tablet_manager.cc:1428] T 07f1cb156e59479eaa1580d7bb2f6e56 P fe94a8257192401c8cc2c45f5acc04d7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:41.493923 8323 tablet_bootstrap.cc:492] T 07f1cb156e59479eaa1580d7bb2f6e56 P 36fdc731da6f4a7da1be563b3720ae12: Bootstrap starting.
I20260619 02:39:41.494307 8323 tablet_bootstrap.cc:654] T 07f1cb156e59479eaa1580d7bb2f6e56 P 36fdc731da6f4a7da1be563b3720ae12: Neither blocks nor log segments found. Creating new log.
I20260619 02:39:41.494792 8094 tablet_service.cc:1467] Processing CreateTablet for tablet 07f1cb156e59479eaa1580d7bb2f6e56 (DEFAULT_TABLE table=test-table [id=bdad03b0b214411dbd462d00b1c2000a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:39:41.494922 8094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07f1cb156e59479eaa1580d7bb2f6e56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:39:41.496732 8323 tablet_bootstrap.cc:492] T 07f1cb156e59479eaa1580d7bb2f6e56 P 36fdc731da6f4a7da1be563b3720ae12: No bootstrap required, opened a new log
I20260619 02:39:41.496786 8323 ts_tablet_manager.cc:1397] T 07f1cb156e59479eaa1580d7bb2f6e56 P 36fdc731da6f4a7da1be563b3720ae12: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260619 02:39:41.496949 8323 raft_consensus.cc:348] T 07f1cb156e59479eaa1580d7bb2f6e56 P 36fdc731da6f4a7da1be563b3720ae12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } }
I20260619 02:39:41.497017 8323 raft_consensus.cc:374] T 07f1cb156e59479eaa1580d7bb2f6e56 P 36fdc731da6f4a7da1be563b3720ae12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:39:41.497040 8323 raft_consensus.cc:729] T 07f1cb156e59479eaa1580d7bb2f6e56 P 36fdc731da6f4a7da1be563b3720ae12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36fdc731da6f4a7da1be563b3720ae12, State: Initialized, Role: FOLLOWER
I20260619 02:39:41.497083 8323 consensus_queue.cc:260] T 07f1cb156e59479eaa1580d7bb2f6e56 P 36fdc731da6f4a7da1be563b3720ae12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } }
I20260619 02:39:41.497184 8323 ts_tablet_manager.cc:1428] T 07f1cb156e59479eaa1580d7bb2f6e56 P 36fdc731da6f4a7da1be563b3720ae12: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:41.498062 8327 tablet_bootstrap.cc:492] T 07f1cb156e59479eaa1580d7bb2f6e56 P 5b1a4ce14d2f4b6d846560a4c3e181bf: Bootstrap starting.
I20260619 02:39:41.498494 8327 tablet_bootstrap.cc:654] T 07f1cb156e59479eaa1580d7bb2f6e56 P 5b1a4ce14d2f4b6d846560a4c3e181bf: Neither blocks nor log segments found. Creating new log.
I20260619 02:39:41.499388 8374 tablet_replica.cc:330] T b099bc5fee544b8c9366ab4a5c667262 P 5b1a4ce14d2f4b6d846560a4c3e181bf: stopping tablet replica
I20260619 02:39:41.499447 8327 tablet_bootstrap.cc:492] T 07f1cb156e59479eaa1580d7bb2f6e56 P 5b1a4ce14d2f4b6d846560a4c3e181bf: No bootstrap required, opened a new log
I20260619 02:39:41.499498 8327 ts_tablet_manager.cc:1397] T 07f1cb156e59479eaa1580d7bb2f6e56 P 5b1a4ce14d2f4b6d846560a4c3e181bf: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.001s
I20260619 02:39:41.499541 8374 raft_consensus.cc:2229] T b099bc5fee544b8c9366ab4a5c667262 P 5b1a4ce14d2f4b6d846560a4c3e181bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:39:41.499621 8374 raft_consensus.cc:2258] T b099bc5fee544b8c9366ab4a5c667262 P 5b1a4ce14d2f4b6d846560a4c3e181bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:39:41.499634 8327 raft_consensus.cc:348] T 07f1cb156e59479eaa1580d7bb2f6e56 P 5b1a4ce14d2f4b6d846560a4c3e181bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } }
I20260619 02:39:41.499707 8327 raft_consensus.cc:374] T 07f1cb156e59479eaa1580d7bb2f6e56 P 5b1a4ce14d2f4b6d846560a4c3e181bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:39:41.499727 8327 raft_consensus.cc:729] T 07f1cb156e59479eaa1580d7bb2f6e56 P 5b1a4ce14d2f4b6d846560a4c3e181bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b1a4ce14d2f4b6d846560a4c3e181bf, State: Initialized, Role: FOLLOWER
I20260619 02:39:41.499770 8327 consensus_queue.cc:260] T 07f1cb156e59479eaa1580d7bb2f6e56 P 5b1a4ce14d2f4b6d846560a4c3e181bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } }
I20260619 02:39:41.499852 8327 ts_tablet_manager.cc:1428] T 07f1cb156e59479eaa1580d7bb2f6e56 P 5b1a4ce14d2f4b6d846560a4c3e181bf: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:41.500700 8374 ts_tablet_manager.cc:1905] T b099bc5fee544b8c9366ab4a5c667262 P 5b1a4ce14d2f4b6d846560a4c3e181bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:39:41.502383 8372 tablet_replica.cc:330] T b099bc5fee544b8c9366ab4a5c667262 P fe94a8257192401c8cc2c45f5acc04d7: stopping tablet replica
I20260619 02:39:41.502521 8372 raft_consensus.cc:2229] T b099bc5fee544b8c9366ab4a5c667262 P fe94a8257192401c8cc2c45f5acc04d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:39:41.502578 8372 pending_rounds.cc:62] T b099bc5fee544b8c9366ab4a5c667262 P fe94a8257192401c8cc2c45f5acc04d7: Trying to abort 2 pending ops.
I20260619 02:39:41.502596 8372 pending_rounds.cc:68] T b099bc5fee544b8c9366ab4a5c667262 P fe94a8257192401c8cc2c45f5acc04d7: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7298403456897572864 op_type: PARTICIPANT_OP participant_request { tablet_id: "b099bc5fee544b8c9366ab4a5c667262" op { type: BEGIN_TXN txn_id: 0 } }
I20260619 02:39:41.502701 8372 pending_rounds.cc:68] T b099bc5fee544b8c9366ab4a5c667262 P fe94a8257192401c8cc2c45f5acc04d7: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7298403455462551552 op_type: NO_OP noop_request { }
I20260619 02:39:41.502744 8372 raft_consensus.cc:2866] T b099bc5fee544b8c9366ab4a5c667262 P fe94a8257192401c8cc2c45f5acc04d7 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260619 02:39:41.502794 8372 raft_consensus.cc:2258] T b099bc5fee544b8c9366ab4a5c667262 P fe94a8257192401c8cc2c45f5acc04d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:39:41.502995 8374 ts_tablet_manager.cc:1918] T b099bc5fee544b8c9366ab4a5c667262 P 5b1a4ce14d2f4b6d846560a4c3e181bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:39:41.503067 8374 log.cc:1196] T b099bc5fee544b8c9366ab4a5c667262 P 5b1a4ce14d2f4b6d846560a4c3e181bf: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-1/wal/wals/b099bc5fee544b8c9366ab4a5c667262
I20260619 02:39:41.503360 8374 ts_tablet_manager.cc:1939] T b099bc5fee544b8c9366ab4a5c667262 P 5b1a4ce14d2f4b6d846560a4c3e181bf: Deleting consensus metadata
I20260619 02:39:41.503756 7801 catalog_manager.cc:4841] TS 5b1a4ce14d2f4b6d846560a4c3e181bf (127.5.241.66:37463): tablet b099bc5fee544b8c9366ab4a5c667262 (table test-table [id=f4736d9c3e1f468294fdcc35d22e9fee]) successfully deleted
I20260619 02:39:41.507076 8372 ts_tablet_manager.cc:1905] T b099bc5fee544b8c9366ab4a5c667262 P fe94a8257192401c8cc2c45f5acc04d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:39:41.508687 8372 ts_tablet_manager.cc:1918] T b099bc5fee544b8c9366ab4a5c667262 P fe94a8257192401c8cc2c45f5acc04d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:39:41.508769 8372 log.cc:1196] T b099bc5fee544b8c9366ab4a5c667262 P fe94a8257192401c8cc2c45f5acc04d7: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-0/wal/wals/b099bc5fee544b8c9366ab4a5c667262
I20260619 02:39:41.509042 8372 ts_tablet_manager.cc:1939] T b099bc5fee544b8c9366ab4a5c667262 P fe94a8257192401c8cc2c45f5acc04d7: Deleting consensus metadata
I20260619 02:39:41.509500 7804 catalog_manager.cc:4841] TS fe94a8257192401c8cc2c45f5acc04d7 (127.5.241.65:40611): tablet b099bc5fee544b8c9366ab4a5c667262 (table test-table [id=f4736d9c3e1f468294fdcc35d22e9fee]) successfully deleted
I20260619 02:39:41.518610 8333 raft_consensus.cc:482] T 07f1cb156e59479eaa1580d7bb2f6e56 P 36fdc731da6f4a7da1be563b3720ae12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:39:41.518707 8333 raft_consensus.cc:504] T 07f1cb156e59479eaa1580d7bb2f6e56 P 36fdc731da6f4a7da1be563b3720ae12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } }
I20260619 02:39:41.518911 8333 leader_election.cc:288] T 07f1cb156e59479eaa1580d7bb2f6e56 P 36fdc731da6f4a7da1be563b3720ae12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fe94a8257192401c8cc2c45f5acc04d7 (127.5.241.65:40611), 5b1a4ce14d2f4b6d846560a4c3e181bf (127.5.241.66:37463)
I20260619 02:39:41.519120 8114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07f1cb156e59479eaa1580d7bb2f6e56" candidate_uuid: "36fdc731da6f4a7da1be563b3720ae12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" is_pre_election: true
I20260619 02:39:41.519177 7974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07f1cb156e59479eaa1580d7bb2f6e56" candidate_uuid: "36fdc731da6f4a7da1be563b3720ae12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe94a8257192401c8cc2c45f5acc04d7" is_pre_election: true
I20260619 02:39:41.519220 8114 raft_consensus.cc:2454] T 07f1cb156e59479eaa1580d7bb2f6e56 P 5b1a4ce14d2f4b6d846560a4c3e181bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36fdc731da6f4a7da1be563b3720ae12 in term 0.
I20260619 02:39:41.519274 7974 raft_consensus.cc:2454] T 07f1cb156e59479eaa1580d7bb2f6e56 P fe94a8257192401c8cc2c45f5acc04d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36fdc731da6f4a7da1be563b3720ae12 in term 0.
I20260619 02:39:41.519435 8185 leader_election.cc:302] T 07f1cb156e59479eaa1580d7bb2f6e56 P 36fdc731da6f4a7da1be563b3720ae12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36fdc731da6f4a7da1be563b3720ae12, 5b1a4ce14d2f4b6d846560a4c3e181bf; no voters:
I20260619 02:39:41.519562 8333 raft_consensus.cc:2781] T 07f1cb156e59479eaa1580d7bb2f6e56 P 36fdc731da6f4a7da1be563b3720ae12 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:39:41.519598 8333 raft_consensus.cc:482] T 07f1cb156e59479eaa1580d7bb2f6e56 P 36fdc731da6f4a7da1be563b3720ae12 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:39:41.519625 8333 raft_consensus.cc:3037] T 07f1cb156e59479eaa1580d7bb2f6e56 P 36fdc731da6f4a7da1be563b3720ae12 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:39:41.520867 8333 raft_consensus.cc:504] T 07f1cb156e59479eaa1580d7bb2f6e56 P 36fdc731da6f4a7da1be563b3720ae12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } }
I20260619 02:39:41.521010 8333 leader_election.cc:288] T 07f1cb156e59479eaa1580d7bb2f6e56 P 36fdc731da6f4a7da1be563b3720ae12 [CANDIDATE]: Term 1 election: Requested vote from peers fe94a8257192401c8cc2c45f5acc04d7 (127.5.241.65:40611), 5b1a4ce14d2f4b6d846560a4c3e181bf (127.5.241.66:37463)
I20260619 02:39:41.521211 7974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07f1cb156e59479eaa1580d7bb2f6e56" candidate_uuid: "36fdc731da6f4a7da1be563b3720ae12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe94a8257192401c8cc2c45f5acc04d7"
I20260619 02:39:41.521234 8114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07f1cb156e59479eaa1580d7bb2f6e56" candidate_uuid: "36fdc731da6f4a7da1be563b3720ae12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf"
I20260619 02:39:41.521297 7974 raft_consensus.cc:3037] T 07f1cb156e59479eaa1580d7bb2f6e56 P fe94a8257192401c8cc2c45f5acc04d7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:39:41.521306 8114 raft_consensus.cc:3037] T 07f1cb156e59479eaa1580d7bb2f6e56 P 5b1a4ce14d2f4b6d846560a4c3e181bf [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:39:41.522049 8114 raft_consensus.cc:2454] T 07f1cb156e59479eaa1580d7bb2f6e56 P 5b1a4ce14d2f4b6d846560a4c3e181bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36fdc731da6f4a7da1be563b3720ae12 in term 1.
I20260619 02:39:41.522097 7974 raft_consensus.cc:2454] T 07f1cb156e59479eaa1580d7bb2f6e56 P fe94a8257192401c8cc2c45f5acc04d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36fdc731da6f4a7da1be563b3720ae12 in term 1.
I20260619 02:39:41.522210 8185 leader_election.cc:302] T 07f1cb156e59479eaa1580d7bb2f6e56 P 36fdc731da6f4a7da1be563b3720ae12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36fdc731da6f4a7da1be563b3720ae12, 5b1a4ce14d2f4b6d846560a4c3e181bf; no voters:
I20260619 02:39:41.522317 8333 raft_consensus.cc:2781] T 07f1cb156e59479eaa1580d7bb2f6e56 P 36fdc731da6f4a7da1be563b3720ae12 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:39:41.522359 8333 raft_consensus.cc:686] T 07f1cb156e59479eaa1580d7bb2f6e56 P 36fdc731da6f4a7da1be563b3720ae12 [term 1 LEADER]: Becoming Leader. State: Replica: 36fdc731da6f4a7da1be563b3720ae12, State: Running, Role: LEADER
I20260619 02:39:41.522399 8333 consensus_queue.cc:237] T 07f1cb156e59479eaa1580d7bb2f6e56 P 36fdc731da6f4a7da1be563b3720ae12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } }
I20260619 02:39:41.522965 7815 catalog_manager.cc:5495] T 07f1cb156e59479eaa1580d7bb2f6e56 P 36fdc731da6f4a7da1be563b3720ae12 reported cstate change: term changed from 0 to 1, leader changed from <none> to 36fdc731da6f4a7da1be563b3720ae12 (127.5.241.67). New cstate: current_term: 1 leader_uuid: "36fdc731da6f4a7da1be563b3720ae12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } health_report { overall_health: HEALTHY } } }
I20260619 02:39:41.530505 8114 raft_consensus.cc:1261] T 07f1cb156e59479eaa1580d7bb2f6e56 P 5b1a4ce14d2f4b6d846560a4c3e181bf [term 1 FOLLOWER]: Refusing update from remote peer 36fdc731da6f4a7da1be563b3720ae12: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:39:41.530684 8328 consensus_queue.cc:1035] T 07f1cb156e59479eaa1580d7bb2f6e56 P 36fdc731da6f4a7da1be563b3720ae12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:39:41.531424 7974 raft_consensus.cc:1261] T 07f1cb156e59479eaa1580d7bb2f6e56 P fe94a8257192401c8cc2c45f5acc04d7 [term 1 FOLLOWER]: Refusing update from remote peer 36fdc731da6f4a7da1be563b3720ae12: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:39:41.531620 8333 consensus_queue.cc:1035] T 07f1cb156e59479eaa1580d7bb2f6e56 P 36fdc731da6f4a7da1be563b3720ae12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 02:39:41.536186 8354 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260619 02:39:43.055660 7817 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56822:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:39:43.055773 7817 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56822:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:39:43.056676 7817 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 61c08775450f47beb0d8ab94784cf7ab: Sending DeleteTablet for 3 replicas of tablet 07f1cb156e59479eaa1580d7bb2f6e56
I20260619 02:39:43.057076 7954 tablet_service.cc:1514] Processing DeleteTablet for tablet 07f1cb156e59479eaa1580d7bb2f6e56 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:39:43 UTC) from {username='kudu', principal='kudu/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:40440
I20260619 02:39:43.057139 8234 tablet_service.cc:1514] Processing DeleteTablet for tablet 07f1cb156e59479eaa1580d7bb2f6e56 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:39:43 UTC) from {username='kudu', principal='kudu/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:59452
I20260619 02:39:43.057170 8094 tablet_service.cc:1514] Processing DeleteTablet for tablet 07f1cb156e59479eaa1580d7bb2f6e56 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:39:43 UTC) from {username='kudu', principal='kudu/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:35128
I20260619 02:39:43.057375 8406 tablet_replica.cc:330] T 07f1cb156e59479eaa1580d7bb2f6e56 P 5b1a4ce14d2f4b6d846560a4c3e181bf: stopping tablet replica
I20260619 02:39:43.057441 8406 raft_consensus.cc:2229] T 07f1cb156e59479eaa1580d7bb2f6e56 P 5b1a4ce14d2f4b6d846560a4c3e181bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:39:43.057485 8406 raft_consensus.cc:2258] T 07f1cb156e59479eaa1580d7bb2f6e56 P 5b1a4ce14d2f4b6d846560a4c3e181bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:39:43.057703 8406 ts_tablet_manager.cc:1905] T 07f1cb156e59479eaa1580d7bb2f6e56 P 5b1a4ce14d2f4b6d846560a4c3e181bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:39:43.059719 8406 ts_tablet_manager.cc:1918] T 07f1cb156e59479eaa1580d7bb2f6e56 P 5b1a4ce14d2f4b6d846560a4c3e181bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 02:39:43.059782 8406 log.cc:1196] T 07f1cb156e59479eaa1580d7bb2f6e56 P 5b1a4ce14d2f4b6d846560a4c3e181bf: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-1/wal/wals/07f1cb156e59479eaa1580d7bb2f6e56
I20260619 02:39:43.060010 8406 ts_tablet_manager.cc:1939] T 07f1cb156e59479eaa1580d7bb2f6e56 P 5b1a4ce14d2f4b6d846560a4c3e181bf: Deleting consensus metadata
I20260619 02:39:43.060468 7801 catalog_manager.cc:4841] TS 5b1a4ce14d2f4b6d846560a4c3e181bf (127.5.241.66:37463): tablet 07f1cb156e59479eaa1580d7bb2f6e56 (table test-table [id=bdad03b0b214411dbd462d00b1c2000a]) successfully deleted
I20260619 02:39:43.062193 8404 tablet_replica.cc:330] T 07f1cb156e59479eaa1580d7bb2f6e56 P fe94a8257192401c8cc2c45f5acc04d7: stopping tablet replica
I20260619 02:39:43.062248 8404 raft_consensus.cc:2229] T 07f1cb156e59479eaa1580d7bb2f6e56 P fe94a8257192401c8cc2c45f5acc04d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:39:43.062304 8404 raft_consensus.cc:2258] T 07f1cb156e59479eaa1580d7bb2f6e56 P fe94a8257192401c8cc2c45f5acc04d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:39:43.062427 8405 tablet_replica.cc:330] T 07f1cb156e59479eaa1580d7bb2f6e56 P 36fdc731da6f4a7da1be563b3720ae12: stopping tablet replica
I20260619 02:39:43.062490 8405 raft_consensus.cc:2229] T 07f1cb156e59479eaa1580d7bb2f6e56 P 36fdc731da6f4a7da1be563b3720ae12 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:39:43.062561 8405 raft_consensus.cc:2258] T 07f1cb156e59479eaa1580d7bb2f6e56 P 36fdc731da6f4a7da1be563b3720ae12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:39:43.062600 8404 ts_tablet_manager.cc:1905] T 07f1cb156e59479eaa1580d7bb2f6e56 P fe94a8257192401c8cc2c45f5acc04d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:39:43.062855 8405 ts_tablet_manager.cc:1905] T 07f1cb156e59479eaa1580d7bb2f6e56 P 36fdc731da6f4a7da1be563b3720ae12: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:39:43.064440 8404 ts_tablet_manager.cc:1918] T 07f1cb156e59479eaa1580d7bb2f6e56 P fe94a8257192401c8cc2c45f5acc04d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 02:39:43.064508 8404 log.cc:1196] T 07f1cb156e59479eaa1580d7bb2f6e56 P fe94a8257192401c8cc2c45f5acc04d7: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-0/wal/wals/07f1cb156e59479eaa1580d7bb2f6e56
I20260619 02:39:43.064575 8405 ts_tablet_manager.cc:1918] T 07f1cb156e59479eaa1580d7bb2f6e56 P 36fdc731da6f4a7da1be563b3720ae12: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 02:39:43.064631 8405 log.cc:1196] T 07f1cb156e59479eaa1580d7bb2f6e56 P 36fdc731da6f4a7da1be563b3720ae12: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-2/wal/wals/07f1cb156e59479eaa1580d7bb2f6e56
I20260619 02:39:43.064766 8404 ts_tablet_manager.cc:1939] T 07f1cb156e59479eaa1580d7bb2f6e56 P fe94a8257192401c8cc2c45f5acc04d7: Deleting consensus metadata
I20260619 02:39:43.065085 7804 catalog_manager.cc:4841] TS fe94a8257192401c8cc2c45f5acc04d7 (127.5.241.65:40611): tablet 07f1cb156e59479eaa1580d7bb2f6e56 (table test-table [id=bdad03b0b214411dbd462d00b1c2000a]) successfully deleted
I20260619 02:39:43.068601 8405 ts_tablet_manager.cc:1939] T 07f1cb156e59479eaa1580d7bb2f6e56 P 36fdc731da6f4a7da1be563b3720ae12: Deleting consensus metadata
Jun 19 02:39:43 dist-test-slave-zpbg krb5kdc[7772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836783, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
I20260619 02:39:43.068972 7801 catalog_manager.cc:4841] TS 36fdc731da6f4a7da1be563b3720ae12 (127.5.241.67:38815): tablet 07f1cb156e59479eaa1580d7bb2f6e56 (table test-table [id=bdad03b0b214411dbd462d00b1c2000a]) successfully deleted
Jun 19 02:39:43 dist-test-slave-zpbg krb5kdc[7772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836783, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:39:43.081907 8414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:43.074394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56840 (local address 127.5.241.126:46063)
0619 02:39:43.074521 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:43.074523 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:43.074544 (+ 21us) server_negotiation.cc:407] Connection header received
0619 02:39:43.074608 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:43.074609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:43.074619 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:43.074679 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:43.075297 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:43.075802 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:43.076586 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:43.076799 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:43.076862 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:43.079255 (+ 2393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:43.079257 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:43.079257 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:43.079264 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:43.080869 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:43.081184 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:43.081185 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:43.081186 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:43.081240 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:43.081439 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:43.081440 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:43.081440 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:43.081582 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:43.081660 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:43.081761 (+ 101us) server_negotiation.cc:299] Negotiation successful
0619 02:39:43.081839 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":75,"threads_started":1}
I20260619 02:39:43.083688 7817 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56840:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:39:43.083863 7817 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:39:43.085407 7954 tablet_service.cc:1467] Processing CreateTablet for tablet 5e0afb49d6584682870326088b0e77e4 (DEFAULT_TABLE table=test-table [id=0eef5a2c58af43e2ac7e92aa225a31ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:39:43.085448 8094 tablet_service.cc:1467] Processing CreateTablet for tablet 5e0afb49d6584682870326088b0e77e4 (DEFAULT_TABLE table=test-table [id=0eef5a2c58af43e2ac7e92aa225a31ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:39:43.085469 8234 tablet_service.cc:1467] Processing CreateTablet for tablet 5e0afb49d6584682870326088b0e77e4 (DEFAULT_TABLE table=test-table [id=0eef5a2c58af43e2ac7e92aa225a31ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:39:43.085547 8094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e0afb49d6584682870326088b0e77e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:39:43.085547 7954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e0afb49d6584682870326088b0e77e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:39:43.085613 8234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e0afb49d6584682870326088b0e77e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:39:43.086738 8415 tablet_bootstrap.cc:492] T 5e0afb49d6584682870326088b0e77e4 P 36fdc731da6f4a7da1be563b3720ae12: Bootstrap starting.
I20260619 02:39:43.087009 8416 tablet_bootstrap.cc:492] T 5e0afb49d6584682870326088b0e77e4 P 5b1a4ce14d2f4b6d846560a4c3e181bf: Bootstrap starting.
I20260619 02:39:43.087052 8417 tablet_bootstrap.cc:492] T 5e0afb49d6584682870326088b0e77e4 P fe94a8257192401c8cc2c45f5acc04d7: Bootstrap starting.
I20260619 02:39:43.087330 8415 tablet_bootstrap.cc:654] T 5e0afb49d6584682870326088b0e77e4 P 36fdc731da6f4a7da1be563b3720ae12: Neither blocks nor log segments found. Creating new log.
I20260619 02:39:43.087368 8416 tablet_bootstrap.cc:654] T 5e0afb49d6584682870326088b0e77e4 P 5b1a4ce14d2f4b6d846560a4c3e181bf: Neither blocks nor log segments found. Creating new log.
I20260619 02:39:43.087386 8417 tablet_bootstrap.cc:654] T 5e0afb49d6584682870326088b0e77e4 P fe94a8257192401c8cc2c45f5acc04d7: Neither blocks nor log segments found. Creating new log.
I20260619 02:39:43.087970 8415 tablet_bootstrap.cc:492] T 5e0afb49d6584682870326088b0e77e4 P 36fdc731da6f4a7da1be563b3720ae12: No bootstrap required, opened a new log
I20260619 02:39:43.088011 8415 ts_tablet_manager.cc:1397] T 5e0afb49d6584682870326088b0e77e4 P 36fdc731da6f4a7da1be563b3720ae12: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:39:43.088042 8416 tablet_bootstrap.cc:492] T 5e0afb49d6584682870326088b0e77e4 P 5b1a4ce14d2f4b6d846560a4c3e181bf: No bootstrap required, opened a new log
I20260619 02:39:43.088094 8416 ts_tablet_manager.cc:1397] T 5e0afb49d6584682870326088b0e77e4 P 5b1a4ce14d2f4b6d846560a4c3e181bf: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:39:43.088142 8415 raft_consensus.cc:348] T 5e0afb49d6584682870326088b0e77e4 P 36fdc731da6f4a7da1be563b3720ae12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } }
I20260619 02:39:43.088219 8415 raft_consensus.cc:374] T 5e0afb49d6584682870326088b0e77e4 P 36fdc731da6f4a7da1be563b3720ae12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:39:43.088239 8415 raft_consensus.cc:729] T 5e0afb49d6584682870326088b0e77e4 P 36fdc731da6f4a7da1be563b3720ae12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36fdc731da6f4a7da1be563b3720ae12, State: Initialized, Role: FOLLOWER
I20260619 02:39:43.088248 8416 raft_consensus.cc:348] T 5e0afb49d6584682870326088b0e77e4 P 5b1a4ce14d2f4b6d846560a4c3e181bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } }
I20260619 02:39:43.088322 8416 raft_consensus.cc:374] T 5e0afb49d6584682870326088b0e77e4 P 5b1a4ce14d2f4b6d846560a4c3e181bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:39:43.088307 8415 consensus_queue.cc:260] T 5e0afb49d6584682870326088b0e77e4 P 36fdc731da6f4a7da1be563b3720ae12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } }
I20260619 02:39:43.088353 8416 raft_consensus.cc:729] T 5e0afb49d6584682870326088b0e77e4 P 5b1a4ce14d2f4b6d846560a4c3e181bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b1a4ce14d2f4b6d846560a4c3e181bf, State: Initialized, Role: FOLLOWER
I20260619 02:39:43.088406 8415 ts_tablet_manager.cc:1428] T 5e0afb49d6584682870326088b0e77e4 P 36fdc731da6f4a7da1be563b3720ae12: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 02:39:43.088408 8416 consensus_queue.cc:260] T 5e0afb49d6584682870326088b0e77e4 P 5b1a4ce14d2f4b6d846560a4c3e181bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } }
I20260619 02:39:43.088533 8416 ts_tablet_manager.cc:1428] T 5e0afb49d6584682870326088b0e77e4 P 5b1a4ce14d2f4b6d846560a4c3e181bf: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 02:39:43.088932 8417 tablet_bootstrap.cc:492] T 5e0afb49d6584682870326088b0e77e4 P fe94a8257192401c8cc2c45f5acc04d7: No bootstrap required, opened a new log
I20260619 02:39:43.088986 8417 ts_tablet_manager.cc:1397] T 5e0afb49d6584682870326088b0e77e4 P fe94a8257192401c8cc2c45f5acc04d7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:39:43.089154 8417 raft_consensus.cc:348] T 5e0afb49d6584682870326088b0e77e4 P fe94a8257192401c8cc2c45f5acc04d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } }
I20260619 02:39:43.089236 8417 raft_consensus.cc:374] T 5e0afb49d6584682870326088b0e77e4 P fe94a8257192401c8cc2c45f5acc04d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:39:43.089268 8417 raft_consensus.cc:729] T 5e0afb49d6584682870326088b0e77e4 P fe94a8257192401c8cc2c45f5acc04d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe94a8257192401c8cc2c45f5acc04d7, State: Initialized, Role: FOLLOWER
I20260619 02:39:43.089323 8417 consensus_queue.cc:260] T 5e0afb49d6584682870326088b0e77e4 P fe94a8257192401c8cc2c45f5acc04d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } }
I20260619 02:39:43.089498 8417 ts_tablet_manager.cc:1428] T 5e0afb49d6584682870326088b0e77e4 P fe94a8257192401c8cc2c45f5acc04d7: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 02:39:43.131986 8385 raft_consensus.cc:482] T 5e0afb49d6584682870326088b0e77e4 P 5b1a4ce14d2f4b6d846560a4c3e181bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:39:43.132104 8385 raft_consensus.cc:504] T 5e0afb49d6584682870326088b0e77e4 P 5b1a4ce14d2f4b6d846560a4c3e181bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } }
I20260619 02:39:43.132294 8385 leader_election.cc:288] T 5e0afb49d6584682870326088b0e77e4 P 5b1a4ce14d2f4b6d846560a4c3e181bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fe94a8257192401c8cc2c45f5acc04d7 (127.5.241.65:40611), 36fdc731da6f4a7da1be563b3720ae12 (127.5.241.67:38815)
I20260619 02:39:43.132475 8254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e0afb49d6584682870326088b0e77e4" candidate_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36fdc731da6f4a7da1be563b3720ae12" is_pre_election: true
I20260619 02:39:43.132586 8254 raft_consensus.cc:2454] T 5e0afb49d6584682870326088b0e77e4 P 36fdc731da6f4a7da1be563b3720ae12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b1a4ce14d2f4b6d846560a4c3e181bf in term 0.
I20260619 02:39:43.132524 7974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e0afb49d6584682870326088b0e77e4" candidate_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe94a8257192401c8cc2c45f5acc04d7" is_pre_election: true
I20260619 02:39:43.132664 7974 raft_consensus.cc:2454] T 5e0afb49d6584682870326088b0e77e4 P fe94a8257192401c8cc2c45f5acc04d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b1a4ce14d2f4b6d846560a4c3e181bf in term 0.
I20260619 02:39:43.132766 8045 leader_election.cc:302] T 5e0afb49d6584682870326088b0e77e4 P 5b1a4ce14d2f4b6d846560a4c3e181bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36fdc731da6f4a7da1be563b3720ae12, 5b1a4ce14d2f4b6d846560a4c3e181bf; no voters:
I20260619 02:39:43.132859 8385 raft_consensus.cc:2781] T 5e0afb49d6584682870326088b0e77e4 P 5b1a4ce14d2f4b6d846560a4c3e181bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:39:43.132899 8385 raft_consensus.cc:482] T 5e0afb49d6584682870326088b0e77e4 P 5b1a4ce14d2f4b6d846560a4c3e181bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:39:43.132933 8385 raft_consensus.cc:3037] T 5e0afb49d6584682870326088b0e77e4 P 5b1a4ce14d2f4b6d846560a4c3e181bf [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:39:43.133626 8385 raft_consensus.cc:504] T 5e0afb49d6584682870326088b0e77e4 P 5b1a4ce14d2f4b6d846560a4c3e181bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } }
I20260619 02:39:43.133750 8385 leader_election.cc:288] T 5e0afb49d6584682870326088b0e77e4 P 5b1a4ce14d2f4b6d846560a4c3e181bf [CANDIDATE]: Term 1 election: Requested vote from peers fe94a8257192401c8cc2c45f5acc04d7 (127.5.241.65:40611), 36fdc731da6f4a7da1be563b3720ae12 (127.5.241.67:38815)
I20260619 02:39:43.133915 8254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e0afb49d6584682870326088b0e77e4" candidate_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36fdc731da6f4a7da1be563b3720ae12"
I20260619 02:39:43.133944 7974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e0afb49d6584682870326088b0e77e4" candidate_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe94a8257192401c8cc2c45f5acc04d7"
I20260619 02:39:43.133999 7974 raft_consensus.cc:3037] T 5e0afb49d6584682870326088b0e77e4 P fe94a8257192401c8cc2c45f5acc04d7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:39:43.134011 8254 raft_consensus.cc:3037] T 5e0afb49d6584682870326088b0e77e4 P 36fdc731da6f4a7da1be563b3720ae12 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:39:43.134598 7974 raft_consensus.cc:2454] T 5e0afb49d6584682870326088b0e77e4 P fe94a8257192401c8cc2c45f5acc04d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b1a4ce14d2f4b6d846560a4c3e181bf in term 1.
I20260619 02:39:43.134606 8254 raft_consensus.cc:2454] T 5e0afb49d6584682870326088b0e77e4 P 36fdc731da6f4a7da1be563b3720ae12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b1a4ce14d2f4b6d846560a4c3e181bf in term 1.
I20260619 02:39:43.134761 8048 leader_election.cc:302] T 5e0afb49d6584682870326088b0e77e4 P 5b1a4ce14d2f4b6d846560a4c3e181bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b1a4ce14d2f4b6d846560a4c3e181bf, fe94a8257192401c8cc2c45f5acc04d7; no voters:
I20260619 02:39:43.134922 8385 raft_consensus.cc:2781] T 5e0afb49d6584682870326088b0e77e4 P 5b1a4ce14d2f4b6d846560a4c3e181bf [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:39:43.134985 8385 raft_consensus.cc:686] T 5e0afb49d6584682870326088b0e77e4 P 5b1a4ce14d2f4b6d846560a4c3e181bf [term 1 LEADER]: Becoming Leader. State: Replica: 5b1a4ce14d2f4b6d846560a4c3e181bf, State: Running, Role: LEADER
I20260619 02:39:43.135040 8385 consensus_queue.cc:237] T 5e0afb49d6584682870326088b0e77e4 P 5b1a4ce14d2f4b6d846560a4c3e181bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } }
I20260619 02:39:43.135717 7815 catalog_manager.cc:5495] T 5e0afb49d6584682870326088b0e77e4 P 5b1a4ce14d2f4b6d846560a4c3e181bf reported cstate change: term changed from 0 to 1, leader changed from <none> to 5b1a4ce14d2f4b6d846560a4c3e181bf (127.5.241.66). New cstate: current_term: 1 leader_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } health_report { overall_health: UNKNOWN } } }
I20260619 02:39:43.150792 8424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:43.148233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35142 (local address 127.5.241.66:37463)
0619 02:39:43.148390 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:43.148392 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:43.148418 (+ 26us) server_negotiation.cc:407] Connection header received
0619 02:39:43.148442 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:43.148442 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:43.148456 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:43.148530 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:39:43.148642 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:43.149140 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:43.150038 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:43.150251 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:43.150335 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:43.150487 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:39:43.150524 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:39:43.150596 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:43.150658 (+ 62us) server_negotiation.cc:299] Negotiation successful
0619 02:39:43.150679 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":86,"threads_started":1}
I20260619 02:39:43.156517 8427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:43.152930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.66:37463 (local address 127.5.241.66:38825)
0619 02:39:43.153082 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:43.153093 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:43.153151 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:43.153288 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:43.153289 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:43.153291 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:39:43.153334 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:43.153335 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:43.153967 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:43.153968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:43.155135 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:43.155137 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:43.155929 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:43.155930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:43.155976 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:43.155980 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:39:43.156337 (+ 357us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:39:43.156340 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:39:43.156394 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 02:39:43.156434 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":105,"threads_started":1}
I20260619 02:39:43.156636 8424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:43.153043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:38825 (local address 127.5.241.66:37463)
0619 02:39:43.153104 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:43.153105 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:43.153145 (+ 40us) server_negotiation.cc:407] Connection header received
0619 02:39:43.153191 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:43.153192 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:43.153200 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:43.153254 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:39:43.153434 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:43.153884 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:43.155458 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:43.155642 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:43.155711 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:43.156168 (+ 457us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:39:43.156195 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:39:43.156248 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:43.156521 (+ 273us) server_negotiation.cc:299] Negotiation successful
0619 02:39:43.156543 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 02:39:43.159183 8254 raft_consensus.cc:1261] T 5e0afb49d6584682870326088b0e77e4 P 36fdc731da6f4a7da1be563b3720ae12 [term 1 FOLLOWER]: Refusing update from remote peer 5b1a4ce14d2f4b6d846560a4c3e181bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:39:43.159206 7974 raft_consensus.cc:1261] T 5e0afb49d6584682870326088b0e77e4 P fe94a8257192401c8cc2c45f5acc04d7 [term 1 FOLLOWER]: Refusing update from remote peer 5b1a4ce14d2f4b6d846560a4c3e181bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:39:43.159387 8357 consensus_queue.cc:1035] T 5e0afb49d6584682870326088b0e77e4 P 5b1a4ce14d2f4b6d846560a4c3e181bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:39:43.159490 8390 consensus_queue.cc:1035] T 5e0afb49d6584682870326088b0e77e4 P 5b1a4ce14d2f4b6d846560a4c3e181bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:39:43.161002 8386 mvcc.cc:205] Tried to move back new op lower bound from 7298403463818506240 to 7298403463721349120. Current Snapshot: MvccSnapshot[applied={T|T < 7298403463818506240}]
I20260619 02:39:43.169941 7817 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56840:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:39:43.170058 7817 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56840:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:39:43.170689 7817 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 61c08775450f47beb0d8ab94784cf7ab: Sending DeleteTablet for 3 replicas of tablet 5e0afb49d6584682870326088b0e77e4
I20260619 02:39:43.170939 8094 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e0afb49d6584682870326088b0e77e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:39:43 UTC) from {username='kudu', principal='kudu/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:35128
I20260619 02:39:43.170981 7954 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e0afb49d6584682870326088b0e77e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:39:43 UTC) from {username='kudu', principal='kudu/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:40440
I20260619 02:39:43.170998 8234 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e0afb49d6584682870326088b0e77e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:39:43 UTC) from {username='kudu', principal='kudu/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:59452
I20260619 02:39:43.171036 8406 tablet_replica.cc:330] T 5e0afb49d6584682870326088b0e77e4 P 5b1a4ce14d2f4b6d846560a4c3e181bf: stopping tablet replica
I20260619 02:39:43.171082 8405 tablet_replica.cc:330] T 5e0afb49d6584682870326088b0e77e4 P 36fdc731da6f4a7da1be563b3720ae12: stopping tablet replica
I20260619 02:39:43.171087 8406 raft_consensus.cc:2229] T 5e0afb49d6584682870326088b0e77e4 P 5b1a4ce14d2f4b6d846560a4c3e181bf [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:39:43.171128 8405 raft_consensus.cc:2229] T 5e0afb49d6584682870326088b0e77e4 P 36fdc731da6f4a7da1be563b3720ae12 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:39:43.171159 8405 raft_consensus.cc:2258] T 5e0afb49d6584682870326088b0e77e4 P 36fdc731da6f4a7da1be563b3720ae12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:39:43.171170 8406 raft_consensus.cc:2258] T 5e0afb49d6584682870326088b0e77e4 P 5b1a4ce14d2f4b6d846560a4c3e181bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:39:43.171252 8404 tablet_replica.cc:330] T 5e0afb49d6584682870326088b0e77e4 P fe94a8257192401c8cc2c45f5acc04d7: stopping tablet replica
I20260619 02:39:43.171319 8404 raft_consensus.cc:2229] T 5e0afb49d6584682870326088b0e77e4 P fe94a8257192401c8cc2c45f5acc04d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:39:43.171367 8404 raft_consensus.cc:2258] T 5e0afb49d6584682870326088b0e77e4 P fe94a8257192401c8cc2c45f5acc04d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:39:43.171435 7817 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56840:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:39:43.171586 7817 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:39:43.171635 8405 ts_tablet_manager.cc:1905] T 5e0afb49d6584682870326088b0e77e4 P 36fdc731da6f4a7da1be563b3720ae12: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:39:43.171677 8406 ts_tablet_manager.cc:1905] T 5e0afb49d6584682870326088b0e77e4 P 5b1a4ce14d2f4b6d846560a4c3e181bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:39:43.173141 8406 ts_tablet_manager.cc:1918] T 5e0afb49d6584682870326088b0e77e4 P 5b1a4ce14d2f4b6d846560a4c3e181bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:39:43.173208 8406 log.cc:1196] T 5e0afb49d6584682870326088b0e77e4 P 5b1a4ce14d2f4b6d846560a4c3e181bf: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-1/wal/wals/5e0afb49d6584682870326088b0e77e4
I20260619 02:39:43.173218 8405 ts_tablet_manager.cc:1918] T 5e0afb49d6584682870326088b0e77e4 P 36fdc731da6f4a7da1be563b3720ae12: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:39:43.173265 8405 log.cc:1196] T 5e0afb49d6584682870326088b0e77e4 P 36fdc731da6f4a7da1be563b3720ae12: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-2/wal/wals/5e0afb49d6584682870326088b0e77e4
I20260619 02:39:43.173437 8406 ts_tablet_manager.cc:1939] T 5e0afb49d6584682870326088b0e77e4 P 5b1a4ce14d2f4b6d846560a4c3e181bf: Deleting consensus metadata
I20260619 02:39:43.173528 8405 ts_tablet_manager.cc:1939] T 5e0afb49d6584682870326088b0e77e4 P 36fdc731da6f4a7da1be563b3720ae12: Deleting consensus metadata
I20260619 02:39:43.173794 7954 tablet_service.cc:1467] Processing CreateTablet for tablet a5ec96eb0c494d2bb3b8414554ecdd41 (DEFAULT_TABLE table=test-table [id=ce9c7deea89444c5b8b9331d8dabd312]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:39:43.173900 7954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5ec96eb0c494d2bb3b8414554ecdd41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:39:43.173921 8404 ts_tablet_manager.cc:1905] T 5e0afb49d6584682870326088b0e77e4 P fe94a8257192401c8cc2c45f5acc04d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:39:43.174942 8417 tablet_bootstrap.cc:492] T a5ec96eb0c494d2bb3b8414554ecdd41 P fe94a8257192401c8cc2c45f5acc04d7: Bootstrap starting.
I20260619 02:39:43.175218 7801 catalog_manager.cc:4841] TS 36fdc731da6f4a7da1be563b3720ae12 (127.5.241.67:38815): tablet 5e0afb49d6584682870326088b0e77e4 (table test-table [id=0eef5a2c58af43e2ac7e92aa225a31ad]) successfully deleted
I20260619 02:39:43.175263 8404 ts_tablet_manager.cc:1918] T 5e0afb49d6584682870326088b0e77e4 P fe94a8257192401c8cc2c45f5acc04d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:39:43.175282 7801 catalog_manager.cc:4841] TS 5b1a4ce14d2f4b6d846560a4c3e181bf (127.5.241.66:37463): tablet 5e0afb49d6584682870326088b0e77e4 (table test-table [id=0eef5a2c58af43e2ac7e92aa225a31ad]) successfully deleted
I20260619 02:39:43.175311 8404 log.cc:1196] T 5e0afb49d6584682870326088b0e77e4 P fe94a8257192401c8cc2c45f5acc04d7: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-0/wal/wals/5e0afb49d6584682870326088b0e77e4
I20260619 02:39:43.175414 8234 tablet_service.cc:1467] Processing CreateTablet for tablet a5ec96eb0c494d2bb3b8414554ecdd41 (DEFAULT_TABLE table=test-table [id=ce9c7deea89444c5b8b9331d8dabd312]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:39:43.175503 8234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5ec96eb0c494d2bb3b8414554ecdd41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:39:43.175544 8417 tablet_bootstrap.cc:654] T a5ec96eb0c494d2bb3b8414554ecdd41 P fe94a8257192401c8cc2c45f5acc04d7: Neither blocks nor log segments found. Creating new log.
I20260619 02:39:43.175549 8404 ts_tablet_manager.cc:1939] T 5e0afb49d6584682870326088b0e77e4 P fe94a8257192401c8cc2c45f5acc04d7: Deleting consensus metadata
I20260619 02:39:43.176170 8417 tablet_bootstrap.cc:492] T a5ec96eb0c494d2bb3b8414554ecdd41 P fe94a8257192401c8cc2c45f5acc04d7: No bootstrap required, opened a new log
I20260619 02:39:43.176213 8417 ts_tablet_manager.cc:1397] T a5ec96eb0c494d2bb3b8414554ecdd41 P fe94a8257192401c8cc2c45f5acc04d7: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:39:43.176355 8417 raft_consensus.cc:348] T a5ec96eb0c494d2bb3b8414554ecdd41 P fe94a8257192401c8cc2c45f5acc04d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } }
I20260619 02:39:43.176594 8417 raft_consensus.cc:374] T a5ec96eb0c494d2bb3b8414554ecdd41 P fe94a8257192401c8cc2c45f5acc04d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:39:43.176641 8417 raft_consensus.cc:729] T a5ec96eb0c494d2bb3b8414554ecdd41 P fe94a8257192401c8cc2c45f5acc04d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe94a8257192401c8cc2c45f5acc04d7, State: Initialized, Role: FOLLOWER
I20260619 02:39:43.176703 8417 consensus_queue.cc:260] T a5ec96eb0c494d2bb3b8414554ecdd41 P fe94a8257192401c8cc2c45f5acc04d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } }
I20260619 02:39:43.176805 8415 tablet_bootstrap.cc:492] T a5ec96eb0c494d2bb3b8414554ecdd41 P 36fdc731da6f4a7da1be563b3720ae12: Bootstrap starting.
I20260619 02:39:43.176826 8417 ts_tablet_manager.cc:1428] T a5ec96eb0c494d2bb3b8414554ecdd41 P fe94a8257192401c8cc2c45f5acc04d7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:39:43.177028 7804 catalog_manager.cc:4841] TS fe94a8257192401c8cc2c45f5acc04d7 (127.5.241.65:40611): tablet 5e0afb49d6584682870326088b0e77e4 (table test-table [id=0eef5a2c58af43e2ac7e92aa225a31ad]) successfully deleted
I20260619 02:39:43.177378 8415 tablet_bootstrap.cc:654] T a5ec96eb0c494d2bb3b8414554ecdd41 P 36fdc731da6f4a7da1be563b3720ae12: Neither blocks nor log segments found. Creating new log.
I20260619 02:39:43.177971 8094 tablet_service.cc:1467] Processing CreateTablet for tablet a5ec96eb0c494d2bb3b8414554ecdd41 (DEFAULT_TABLE table=test-table [id=ce9c7deea89444c5b8b9331d8dabd312]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:39:43.178022 8415 tablet_bootstrap.cc:492] T a5ec96eb0c494d2bb3b8414554ecdd41 P 36fdc731da6f4a7da1be563b3720ae12: No bootstrap required, opened a new log
I20260619 02:39:43.178078 8094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5ec96eb0c494d2bb3b8414554ecdd41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:39:43.178094 8415 ts_tablet_manager.cc:1397] T a5ec96eb0c494d2bb3b8414554ecdd41 P 36fdc731da6f4a7da1be563b3720ae12: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:39:43.178195 8415 raft_consensus.cc:348] T a5ec96eb0c494d2bb3b8414554ecdd41 P 36fdc731da6f4a7da1be563b3720ae12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } }
I20260619 02:39:43.178277 8415 raft_consensus.cc:374] T a5ec96eb0c494d2bb3b8414554ecdd41 P 36fdc731da6f4a7da1be563b3720ae12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:39:43.178306 8415 raft_consensus.cc:729] T a5ec96eb0c494d2bb3b8414554ecdd41 P 36fdc731da6f4a7da1be563b3720ae12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36fdc731da6f4a7da1be563b3720ae12, State: Initialized, Role: FOLLOWER
I20260619 02:39:43.178354 8415 consensus_queue.cc:260] T a5ec96eb0c494d2bb3b8414554ecdd41 P 36fdc731da6f4a7da1be563b3720ae12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } }
I20260619 02:39:43.178464 8415 ts_tablet_manager.cc:1428] T a5ec96eb0c494d2bb3b8414554ecdd41 P 36fdc731da6f4a7da1be563b3720ae12: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:43.179359 8416 tablet_bootstrap.cc:492] T a5ec96eb0c494d2bb3b8414554ecdd41 P 5b1a4ce14d2f4b6d846560a4c3e181bf: Bootstrap starting.
I20260619 02:39:43.179664 8416 tablet_bootstrap.cc:654] T a5ec96eb0c494d2bb3b8414554ecdd41 P 5b1a4ce14d2f4b6d846560a4c3e181bf: Neither blocks nor log segments found. Creating new log.
I20260619 02:39:43.180258 8416 tablet_bootstrap.cc:492] T a5ec96eb0c494d2bb3b8414554ecdd41 P 5b1a4ce14d2f4b6d846560a4c3e181bf: No bootstrap required, opened a new log
I20260619 02:39:43.180339 8416 ts_tablet_manager.cc:1397] T a5ec96eb0c494d2bb3b8414554ecdd41 P 5b1a4ce14d2f4b6d846560a4c3e181bf: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:39:43.180477 8416 raft_consensus.cc:348] T a5ec96eb0c494d2bb3b8414554ecdd41 P 5b1a4ce14d2f4b6d846560a4c3e181bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } }
I20260619 02:39:43.180569 8416 raft_consensus.cc:374] T a5ec96eb0c494d2bb3b8414554ecdd41 P 5b1a4ce14d2f4b6d846560a4c3e181bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:39:43.180598 8416 raft_consensus.cc:729] T a5ec96eb0c494d2bb3b8414554ecdd41 P 5b1a4ce14d2f4b6d846560a4c3e181bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b1a4ce14d2f4b6d846560a4c3e181bf, State: Initialized, Role: FOLLOWER
I20260619 02:39:43.180655 8416 consensus_queue.cc:260] T a5ec96eb0c494d2bb3b8414554ecdd41 P 5b1a4ce14d2f4b6d846560a4c3e181bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } }
I20260619 02:39:43.180779 8416 ts_tablet_manager.cc:1428] T a5ec96eb0c494d2bb3b8414554ecdd41 P 5b1a4ce14d2f4b6d846560a4c3e181bf: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 02:39:43.264194 8390 raft_consensus.cc:482] T a5ec96eb0c494d2bb3b8414554ecdd41 P 5b1a4ce14d2f4b6d846560a4c3e181bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:39:43.264320 8390 raft_consensus.cc:504] T a5ec96eb0c494d2bb3b8414554ecdd41 P 5b1a4ce14d2f4b6d846560a4c3e181bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } }
I20260619 02:39:43.264480 8390 leader_election.cc:288] T a5ec96eb0c494d2bb3b8414554ecdd41 P 5b1a4ce14d2f4b6d846560a4c3e181bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fe94a8257192401c8cc2c45f5acc04d7 (127.5.241.65:40611), 36fdc731da6f4a7da1be563b3720ae12 (127.5.241.67:38815)
I20260619 02:39:43.264744 7974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5ec96eb0c494d2bb3b8414554ecdd41" candidate_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe94a8257192401c8cc2c45f5acc04d7" is_pre_election: true
I20260619 02:39:43.264741 8254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5ec96eb0c494d2bb3b8414554ecdd41" candidate_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36fdc731da6f4a7da1be563b3720ae12" is_pre_election: true
I20260619 02:39:43.264844 7974 raft_consensus.cc:2454] T a5ec96eb0c494d2bb3b8414554ecdd41 P fe94a8257192401c8cc2c45f5acc04d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b1a4ce14d2f4b6d846560a4c3e181bf in term 0.
I20260619 02:39:43.264865 8254 raft_consensus.cc:2454] T a5ec96eb0c494d2bb3b8414554ecdd41 P 36fdc731da6f4a7da1be563b3720ae12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b1a4ce14d2f4b6d846560a4c3e181bf in term 0.
I20260619 02:39:43.265012 8048 leader_election.cc:302] T a5ec96eb0c494d2bb3b8414554ecdd41 P 5b1a4ce14d2f4b6d846560a4c3e181bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b1a4ce14d2f4b6d846560a4c3e181bf, fe94a8257192401c8cc2c45f5acc04d7; no voters:
I20260619 02:39:43.265097 8390 raft_consensus.cc:2781] T a5ec96eb0c494d2bb3b8414554ecdd41 P 5b1a4ce14d2f4b6d846560a4c3e181bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:39:43.265146 8390 raft_consensus.cc:482] T a5ec96eb0c494d2bb3b8414554ecdd41 P 5b1a4ce14d2f4b6d846560a4c3e181bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:39:43.265169 8390 raft_consensus.cc:3037] T a5ec96eb0c494d2bb3b8414554ecdd41 P 5b1a4ce14d2f4b6d846560a4c3e181bf [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:39:43.266053 8390 raft_consensus.cc:504] T a5ec96eb0c494d2bb3b8414554ecdd41 P 5b1a4ce14d2f4b6d846560a4c3e181bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } }
I20260619 02:39:43.266191 8390 leader_election.cc:288] T a5ec96eb0c494d2bb3b8414554ecdd41 P 5b1a4ce14d2f4b6d846560a4c3e181bf [CANDIDATE]: Term 1 election: Requested vote from peers fe94a8257192401c8cc2c45f5acc04d7 (127.5.241.65:40611), 36fdc731da6f4a7da1be563b3720ae12 (127.5.241.67:38815)
I20260619 02:39:43.266346 7974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5ec96eb0c494d2bb3b8414554ecdd41" candidate_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe94a8257192401c8cc2c45f5acc04d7"
I20260619 02:39:43.266371 8254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5ec96eb0c494d2bb3b8414554ecdd41" candidate_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36fdc731da6f4a7da1be563b3720ae12"
I20260619 02:39:43.266433 8254 raft_consensus.cc:3037] T a5ec96eb0c494d2bb3b8414554ecdd41 P 36fdc731da6f4a7da1be563b3720ae12 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:39:43.266433 7974 raft_consensus.cc:3037] T a5ec96eb0c494d2bb3b8414554ecdd41 P fe94a8257192401c8cc2c45f5acc04d7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:39:43.267191 7974 raft_consensus.cc:2454] T a5ec96eb0c494d2bb3b8414554ecdd41 P fe94a8257192401c8cc2c45f5acc04d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b1a4ce14d2f4b6d846560a4c3e181bf in term 1.
I20260619 02:39:43.267191 8254 raft_consensus.cc:2454] T a5ec96eb0c494d2bb3b8414554ecdd41 P 36fdc731da6f4a7da1be563b3720ae12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b1a4ce14d2f4b6d846560a4c3e181bf in term 1.
I20260619 02:39:43.267367 8048 leader_election.cc:302] T a5ec96eb0c494d2bb3b8414554ecdd41 P 5b1a4ce14d2f4b6d846560a4c3e181bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b1a4ce14d2f4b6d846560a4c3e181bf, fe94a8257192401c8cc2c45f5acc04d7; no voters:
I20260619 02:39:43.267494 8390 raft_consensus.cc:2781] T a5ec96eb0c494d2bb3b8414554ecdd41 P 5b1a4ce14d2f4b6d846560a4c3e181bf [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:39:43.267545 8390 raft_consensus.cc:686] T a5ec96eb0c494d2bb3b8414554ecdd41 P 5b1a4ce14d2f4b6d846560a4c3e181bf [term 1 LEADER]: Becoming Leader. State: Replica: 5b1a4ce14d2f4b6d846560a4c3e181bf, State: Running, Role: LEADER
I20260619 02:39:43.267599 8390 consensus_queue.cc:237] T a5ec96eb0c494d2bb3b8414554ecdd41 P 5b1a4ce14d2f4b6d846560a4c3e181bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } }
I20260619 02:39:43.268246 7815 catalog_manager.cc:5495] T a5ec96eb0c494d2bb3b8414554ecdd41 P 5b1a4ce14d2f4b6d846560a4c3e181bf reported cstate change: term changed from 0 to 1, leader changed from <none> to 5b1a4ce14d2f4b6d846560a4c3e181bf (127.5.241.66). New cstate: current_term: 1 leader_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b1a4ce14d2f4b6d846560a4c3e181bf" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 37463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 } health_report { overall_health: UNKNOWN } } }
I20260619 02:39:43.275805 8254 raft_consensus.cc:1261] T a5ec96eb0c494d2bb3b8414554ecdd41 P 36fdc731da6f4a7da1be563b3720ae12 [term 1 FOLLOWER]: Refusing update from remote peer 5b1a4ce14d2f4b6d846560a4c3e181bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:39:43.275833 7974 raft_consensus.cc:1261] T a5ec96eb0c494d2bb3b8414554ecdd41 P fe94a8257192401c8cc2c45f5acc04d7 [term 1 FOLLOWER]: Refusing update from remote peer 5b1a4ce14d2f4b6d846560a4c3e181bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:39:43.276047 8385 consensus_queue.cc:1035] T a5ec96eb0c494d2bb3b8414554ecdd41 P 5b1a4ce14d2f4b6d846560a4c3e181bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe94a8257192401c8cc2c45f5acc04d7" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:39:43.276129 8390 consensus_queue.cc:1035] T a5ec96eb0c494d2bb3b8414554ecdd41 P 5b1a4ce14d2f4b6d846560a4c3e181bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "36fdc731da6f4a7da1be563b3720ae12" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 02:39:43.280947 8383 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260619 02:39:44.798496 7817 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56840:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:39:44.798612 7817 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56840:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:39:44.799517 7817 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 61c08775450f47beb0d8ab94784cf7ab: Sending DeleteTablet for 3 replicas of tablet a5ec96eb0c494d2bb3b8414554ecdd41
I20260619 02:39:44.799755 8093 tablet_service.cc:1514] Processing DeleteTablet for tablet a5ec96eb0c494d2bb3b8414554ecdd41 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:39:44 UTC) from {username='kudu', principal='kudu/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:35128
I20260619 02:39:44.799793 8234 tablet_service.cc:1514] Processing DeleteTablet for tablet a5ec96eb0c494d2bb3b8414554ecdd41 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:39:44 UTC) from {username='kudu', principal='kudu/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:59452
I20260619 02:39:44.799803 7954 tablet_service.cc:1514] Processing DeleteTablet for tablet a5ec96eb0c494d2bb3b8414554ecdd41 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:39:44 UTC) from {username='kudu', principal='kudu/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:40440
I20260619 02:39:44.799933 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 7889
I20260619 02:39:44.802901 8456 tablet_replica.cc:330] T a5ec96eb0c494d2bb3b8414554ecdd41 P 5b1a4ce14d2f4b6d846560a4c3e181bf: stopping tablet replica
I20260619 02:39:44.802945 8457 tablet_replica.cc:330] T a5ec96eb0c494d2bb3b8414554ecdd41 P 36fdc731da6f4a7da1be563b3720ae12: stopping tablet replica
I20260619 02:39:44.802997 8456 raft_consensus.cc:2229] T a5ec96eb0c494d2bb3b8414554ecdd41 P 5b1a4ce14d2f4b6d846560a4c3e181bf [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:39:44.803011 8457 raft_consensus.cc:2229] T a5ec96eb0c494d2bb3b8414554ecdd41 P 36fdc731da6f4a7da1be563b3720ae12 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:39:44.803062 8457 raft_consensus.cc:2258] T a5ec96eb0c494d2bb3b8414554ecdd41 P 36fdc731da6f4a7da1be563b3720ae12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:39:44.803085 8456 raft_consensus.cc:2258] T a5ec96eb0c494d2bb3b8414554ecdd41 P 5b1a4ce14d2f4b6d846560a4c3e181bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:39:44.803366 8457 ts_tablet_manager.cc:1905] T a5ec96eb0c494d2bb3b8414554ecdd41 P 36fdc731da6f4a7da1be563b3720ae12: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:39:44.803395 8456 ts_tablet_manager.cc:1905] T a5ec96eb0c494d2bb3b8414554ecdd41 P 5b1a4ce14d2f4b6d846560a4c3e181bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:39:44.804965 8456 ts_tablet_manager.cc:1918] T a5ec96eb0c494d2bb3b8414554ecdd41 P 5b1a4ce14d2f4b6d846560a4c3e181bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 02:39:44.805038 8456 log.cc:1196] T a5ec96eb0c494d2bb3b8414554ecdd41 P 5b1a4ce14d2f4b6d846560a4c3e181bf: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-1/wal/wals/a5ec96eb0c494d2bb3b8414554ecdd41
I20260619 02:39:44.805300 8456 ts_tablet_manager.cc:1939] T a5ec96eb0c494d2bb3b8414554ecdd41 P 5b1a4ce14d2f4b6d846560a4c3e181bf: Deleting consensus metadata
I20260619 02:39:44.805311 8457 ts_tablet_manager.cc:1918] T a5ec96eb0c494d2bb3b8414554ecdd41 P 36fdc731da6f4a7da1be563b3720ae12: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 02:39:44.805365 8457 log.cc:1196] T a5ec96eb0c494d2bb3b8414554ecdd41 P 36fdc731da6f4a7da1be563b3720ae12: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836772852275-6085-0/minicluster-data/ts-2/wal/wals/a5ec96eb0c494d2bb3b8414554ecdd41
I20260619 02:39:44.805653 7801 catalog_manager.cc:4841] TS 5b1a4ce14d2f4b6d846560a4c3e181bf (127.5.241.66:37463): tablet a5ec96eb0c494d2bb3b8414554ecdd41 (table test-table [id=ce9c7deea89444c5b8b9331d8dabd312]) successfully deleted
I20260619 02:39:44.805716 8457 ts_tablet_manager.cc:1939] T a5ec96eb0c494d2bb3b8414554ecdd41 P 36fdc731da6f4a7da1be563b3720ae12: Deleting consensus metadata
I20260619 02:39:44.806108 7801 catalog_manager.cc:4841] TS 36fdc731da6f4a7da1be563b3720ae12 (127.5.241.67:38815): tablet a5ec96eb0c494d2bb3b8414554ecdd41 (table test-table [id=ce9c7deea89444c5b8b9331d8dabd312]) successfully deleted
W20260619 02:39:44.806143 7804 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.241.65:40611 (error 108)
I20260619 02:39:44.806331 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 8029
I20260619 02:39:44.811635 8460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:44.806519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:40611 (local address 127.0.0.1:40444)
0619 02:39:44.811521 (+ 5002us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:44.811581 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.65:40611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4967,"thread_start_us":58,"threads_started":1}
W20260619 02:39:44.811796 7804 catalog_manager.cc:4584] TS fe94a8257192401c8cc2c45f5acc04d7 (127.5.241.65:40611): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a5ec96eb0c494d2bb3b8414554ecdd41: Network error: Client connection negotiation failed: client connection to 127.5.241.65:40611: connect: Connection refused (error 111)
I20260619 02:39:44.811914 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 8169
I20260619 02:39:44.817257 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 7785
2026-06-19T02:39:44Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5818 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:39:44 dist-test-slave-zpbg krb5kdc[8463](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:39:44 dist-test-slave-zpbg krb5kdc[8463](info): set up 2 sockets
Jun 19 02:39:44 dist-test-slave-zpbg krb5kdc[8463](info): commencing operation
krb5kdc: starting...
W20260619 02:39:45.855865 6085 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Jun 19 02:39:45 dist-test-slave-zpbg krb5kdc[8463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836785, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:39:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:39:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.241.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
I20260619 02:39:45.977612 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:43007
--webserver_interface=127.5.241.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:38691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:43007
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:39:46.057322 8476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:39:46.057591 8476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:39:46.057633 8476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:39:46.058825 8476 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:39:46.058897 8476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:39:46.058921 8476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:39:46.058943 8476 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:39:46.058970 8476 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:39:46.060493 8476 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:38691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:43007
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:43007
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.241.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8476
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:39:46.060798 8476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:39:46.061033 8476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:39:46.066159 8484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:46.066159 8482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:46.066187 8481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:39:46.066697 8476 server_base.cc:1034] running on GCE node
I20260619 02:39:46.066962 8476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:39:46.067253 8476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:39:46.068455 8476 hybrid_clock.cc:648] HybridClock initialized: now 1781836786068427 us; error 40 us; skew 500 ppm
Jun 19 02:39:46 dist-test-slave-zpbg krb5kdc[8463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836786, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:39:46.071030 8476 init.cc:377] Logged in from keytab as kudu/127.5.241.126@KRBTEST.COM (short username kudu)
I20260619 02:39:46.071499 8476 webserver.cc:460] Webserver started at http://127.5.241.126:36833/ using document root <none> and password file <none>
I20260619 02:39:46.071767 8476 fs_manager.cc:359] Metadata directory not provided
I20260619 02:39:46.071838 8476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:39:46.071988 8476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:39:46.073030 8476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/master-0/data/instance:
uuid: "3a67d8add90d468bbe2485fdb7f992ab"
format_stamp: "Formatted at 2026-06-19 02:39:46 on dist-test-slave-zpbg"
server_key: "3a254fc77f8d33600bb2dca830022367"
server_key_iv: "d8e3dc3f7019acbdec28041ab518edbc"
server_key_version: "encryptionkey@0"
I20260619 02:39:46.073400 8476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/master-0/wal/instance:
uuid: "3a67d8add90d468bbe2485fdb7f992ab"
format_stamp: "Formatted at 2026-06-19 02:39:46 on dist-test-slave-zpbg"
server_key: "3a254fc77f8d33600bb2dca830022367"
server_key_iv: "d8e3dc3f7019acbdec28041ab518edbc"
server_key_version: "encryptionkey@0"
I20260619 02:39:46.074859 8476 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:39:46.075852 8491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:46.076062 8476 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:39:46.076167 8476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "3a67d8add90d468bbe2485fdb7f992ab"
format_stamp: "Formatted at 2026-06-19 02:39:46 on dist-test-slave-zpbg"
server_key: "3a254fc77f8d33600bb2dca830022367"
server_key_iv: "d8e3dc3f7019acbdec28041ab518edbc"
server_key_version: "encryptionkey@0"
I20260619 02:39:46.076258 8476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:39:46.093271 8476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:39:46.095350 8476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:39:46.095515 8476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:39:46.099633 8476 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.126:43007
I20260619 02:39:46.099784 8543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.126:43007 every 8 connection(s)
I20260619 02:39:46.100080 8476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:39:46.100962 8544 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:39:46.102476 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 8476
I20260619 02:39:46.102605 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/master-0/wal/instance
I20260619 02:39:46.102837 6085 external_mini_cluster.cc:1428] Setting key 100f65ed55a7194a2198f6821a28094d
I20260619 02:39:46.103617 8544 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a67d8add90d468bbe2485fdb7f992ab: Bootstrap starting.
I20260619 02:39:46.104224 8544 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3a67d8add90d468bbe2485fdb7f992ab: Neither blocks nor log segments found. Creating new log.
I20260619 02:39:46.104542 8544 log.cc:824] T 00000000000000000000000000000000 P 3a67d8add90d468bbe2485fdb7f992ab: Log is configured to *not* fsync() on all Append() calls
I20260619 02:39:46.105226 8544 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a67d8add90d468bbe2485fdb7f992ab: No bootstrap required, opened a new log
I20260619 02:39:46.107075 8544 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3a67d8add90d468bbe2485fdb7f992ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a67d8add90d468bbe2485fdb7f992ab" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 43007 } }
I20260619 02:39:46.107239 8544 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3a67d8add90d468bbe2485fdb7f992ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:39:46.107300 8544 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3a67d8add90d468bbe2485fdb7f992ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a67d8add90d468bbe2485fdb7f992ab, State: Initialized, Role: FOLLOWER
I20260619 02:39:46.107477 8544 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a67d8add90d468bbe2485fdb7f992ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a67d8add90d468bbe2485fdb7f992ab" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 43007 } }
I20260619 02:39:46.107571 8544 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3a67d8add90d468bbe2485fdb7f992ab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:39:46.107617 8544 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3a67d8add90d468bbe2485fdb7f992ab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:39:46.107666 8544 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3a67d8add90d468bbe2485fdb7f992ab [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:39:46.108196 8544 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3a67d8add90d468bbe2485fdb7f992ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a67d8add90d468bbe2485fdb7f992ab" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 43007 } }
I20260619 02:39:46.108312 8544 leader_election.cc:302] T 00000000000000000000000000000000 P 3a67d8add90d468bbe2485fdb7f992ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3a67d8add90d468bbe2485fdb7f992ab; no voters:
I20260619 02:39:46.108501 8544 leader_election.cc:288] T 00000000000000000000000000000000 P 3a67d8add90d468bbe2485fdb7f992ab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:39:46.108527 8549 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3a67d8add90d468bbe2485fdb7f992ab [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:39:46.108628 8549 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3a67d8add90d468bbe2485fdb7f992ab [term 1 LEADER]: Becoming Leader. State: Replica: 3a67d8add90d468bbe2485fdb7f992ab, State: Running, Role: LEADER
I20260619 02:39:46.108765 8544 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3a67d8add90d468bbe2485fdb7f992ab [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:39:46.108767 8549 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a67d8add90d468bbe2485fdb7f992ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a67d8add90d468bbe2485fdb7f992ab" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 43007 } }
I20260619 02:39:46.109107 8549 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a67d8add90d468bbe2485fdb7f992ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a67d8add90d468bbe2485fdb7f992ab. Latest consensus state: current_term: 1 leader_uuid: "3a67d8add90d468bbe2485fdb7f992ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a67d8add90d468bbe2485fdb7f992ab" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 43007 } } }
I20260619 02:39:46.109189 8549 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a67d8add90d468bbe2485fdb7f992ab [sys.catalog]: This master's current role is: LEADER
Jun 19 02:39:46 dist-test-slave-zpbg krb5kdc[8463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836785, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:39:46.109122 8550 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a67d8add90d468bbe2485fdb7f992ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3a67d8add90d468bbe2485fdb7f992ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a67d8add90d468bbe2485fdb7f992ab" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 43007 } } }
I20260619 02:39:46.109479 8550 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a67d8add90d468bbe2485fdb7f992ab [sys.catalog]: This master's current role is: LEADER
I20260619 02:39:46.109759 8555 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:39:46.110291 8555 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:39:46.112262 8555 catalog_manager.cc:1347] Generated new cluster ID: 60da9254a4da4a90a91c8220583c8197
I20260619 02:39:46.112320 8555 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:39:46.113803 8547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:46.103448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32920 (local address 127.5.241.126:43007)
0619 02:39:46.103621 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:46.103638 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:46.103663 (+ 25us) server_negotiation.cc:407] Connection header received
0619 02:39:46.104145 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:46.104152 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:46.104210 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:46.104366 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:46.105070 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:46.106025 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:46.106802 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:46.107214 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:46.107338 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:46.110099 (+ 2761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:46.110103 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:46.110104 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:46.110116 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:46.112215 (+ 2099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:46.112620 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:46.112623 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:46.112623 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:46.112724 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:46.112932 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:46.112933 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:46.112933 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:46.113180 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:46.113362 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:46.113437 (+ 75us) server_negotiation.cc:299] Negotiation successful
0619 02:39:46.113577 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":97,"threads_started":1}
I20260619 02:39:46.125905 8555 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:39:46.126748 8555 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:39:46.137873 8555 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3a67d8add90d468bbe2485fdb7f992ab: Generated new TSK 0
I20260619 02:39:46.138116 8555 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.241.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab.
Entry for principal kudu/127.5.241.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab.
I20260619 02:39:46.171191 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.65:0
--local_ip_for_outbound_sockets=127.5.241.65
--webserver_interface=127.5.241.65
--webserver_port=0
--tserver_master_addrs=127.5.241.126:43007
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab
--principal=kudu/127.5.241.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:38691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:39:46.257879 8570 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:39:46.258136 8570 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:39:46.258179 8570 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:39:46.259411 8570 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:39:46.259477 8570 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:39:46.259523 8570 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.65
I20260619 02:39:46.261003 8570 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:38691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab
--principal=kudu/127.5.241.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.241.65
--webserver_port=0
--tserver_master_addrs=127.5.241.126:43007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8570
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.65
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:39:46.261270 8570 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:39:46.261509 8570 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:39:46.266477 8575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:46.266444 8578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:46.266551 8576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:39:46.266705 8570 server_base.cc:1034] running on GCE node
I20260619 02:39:46.266957 8570 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:39:46.267227 8570 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:39:46.268410 8570 hybrid_clock.cc:648] HybridClock initialized: now 1781836786268392 us; error 35 us; skew 500 ppm
Jun 19 02:39:46 dist-test-slave-zpbg krb5kdc[8463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836786, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:39:46.271106 8570 init.cc:377] Logged in from keytab as kudu/127.5.241.65@KRBTEST.COM (short username kudu)
I20260619 02:39:46.271565 8570 webserver.cc:460] Webserver started at http://127.5.241.65:33329/ using document root <none> and password file <none>
I20260619 02:39:46.271852 8570 fs_manager.cc:359] Metadata directory not provided
I20260619 02:39:46.271921 8570 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:39:46.272089 8570 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:39:46.273176 8570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-0/data/instance:
uuid: "f11c060bf92b4cc3b9ca6d2bf7838cd9"
format_stamp: "Formatted at 2026-06-19 02:39:46 on dist-test-slave-zpbg"
server_key: "24021914b1e5c82722b2c54334bf9260"
server_key_iv: "90eda1aba40ab381234d9aadb39135ac"
server_key_version: "encryptionkey@0"
I20260619 02:39:46.273551 8570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-0/wal/instance:
uuid: "f11c060bf92b4cc3b9ca6d2bf7838cd9"
format_stamp: "Formatted at 2026-06-19 02:39:46 on dist-test-slave-zpbg"
server_key: "24021914b1e5c82722b2c54334bf9260"
server_key_iv: "90eda1aba40ab381234d9aadb39135ac"
server_key_version: "encryptionkey@0"
I20260619 02:39:46.275173 8570 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 02:39:46.276070 8585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:46.276249 8570 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:39:46.276350 8570 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-0/wal
uuid: "f11c060bf92b4cc3b9ca6d2bf7838cd9"
format_stamp: "Formatted at 2026-06-19 02:39:46 on dist-test-slave-zpbg"
server_key: "24021914b1e5c82722b2c54334bf9260"
server_key_iv: "90eda1aba40ab381234d9aadb39135ac"
server_key_version: "encryptionkey@0"
I20260619 02:39:46.276439 8570 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:39:46.287563 8570 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:39:46.289723 8570 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:39:46.289908 8570 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:39:46.290163 8570 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:39:46.290489 8570 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:39:46.290548 8570 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:46.290594 8570 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:39:46.290625 8570 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:46.297045 8570 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.65:43653
I20260619 02:39:46.297101 8698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.65:43653 every 8 connection(s)
I20260619 02:39:46.297475 8570 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
Jun 19 02:39:46 dist-test-slave-zpbg krb5kdc[8463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836786, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.65@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:39:46.306635 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 8570
I20260619 02:39:46.306733 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-0/wal/instance
I20260619 02:39:46.306931 6085 external_mini_cluster.cc:1428] Setting key 0e28333e9bcfe20d0898ef691e95b84a
I20260619 02:39:46.307320 8547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:46.298335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:54741 (local address 127.5.241.126:43007)
0619 02:39:46.298446 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:46.298448 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:46.298651 (+ 203us) server_negotiation.cc:407] Connection header received
0619 02:39:46.299273 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:46.299275 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:46.299287 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:46.299339 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:46.300310 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:46.300830 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:46.301865 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:46.302065 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:46.302146 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:46.304361 (+ 2215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:46.304363 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:46.304363 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:46.304375 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:46.305673 (+ 1298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:46.306090 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:46.306091 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:46.306091 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:46.306152 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:46.306398 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:46.306399 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:46.306399 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:46.306563 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:46.306639 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:46.307115 (+ 476us) server_negotiation.cc:299] Negotiation successful
0619 02:39:46.307221 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 02:39:46.307552 8701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:46.298294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:43007 (local address 127.5.241.65:54741)
0619 02:39:46.298495 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:46.298543 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:46.299155 (+ 612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:46.299420 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:46.299424 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:46.300016 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:39:46.300207 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:46.300211 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:46.300922 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:46.300923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:46.301757 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:46.301759 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:46.302166 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:46.302167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:46.302294 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:46.302880 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:39:46.302888 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:39:46.304255 (+ 1367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:39:46.305769 (+ 1514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:46.305776 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:46.305778 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:46.305997 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:46.306218 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:46.306219 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:46.306220 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:46.306327 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:46.306682 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:39:46.306693 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:39:46.306917 (+ 224us) client_negotiation.cc:769] Sending connection context
0619 02:39:46.307079 (+ 162us) client_negotiation.cc:240] Negotiation successful
0619 02:39:46.307300 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":88,"threads_started":1}
I20260619 02:39:46.308091 8699 heartbeater.cc:347] Connected to a master server at 127.5.241.126:43007
I20260619 02:39:46.308255 8699 heartbeater.cc:464] Registering TS with master...
I20260619 02:39:46.308460 8699 heartbeater.cc:511] Master 127.5.241.126:43007 requested a full tablet report, sending...
I20260619 02:39:46.309073 8508 ts_manager.cc:194] Registered new tserver with Master: f11c060bf92b4cc3b9ca6d2bf7838cd9 (127.5.241.65:43653)
I20260619 02:39:46.309864 8508 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.65@KRBTEST.COM'} at 127.5.241.65:54741
WARNING: no policy specified for kudu/127.5.241.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab.
Entry for principal kudu/127.5.241.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab.
I20260619 02:39:46.334254 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.66:0
--local_ip_for_outbound_sockets=127.5.241.66
--webserver_interface=127.5.241.66
--webserver_port=0
--tserver_master_addrs=127.5.241.126:43007
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab
--principal=kudu/127.5.241.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:38691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:39:46.414414 8704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:39:46.414625 8704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:39:46.414649 8704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:39:46.415813 8704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:39:46.415858 8704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:39:46.415884 8704 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.66
I20260619 02:39:46.417229 8704 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:38691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab
--principal=kudu/127.5.241.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.241.66
--webserver_port=0
--tserver_master_addrs=127.5.241.126:43007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8704
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.66
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:39:46.417409 8704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:39:46.417593 8704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:39:46.422722 8710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:46.422747 8709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:46.422730 8712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:39:46.423115 8704 server_base.cc:1034] running on GCE node
I20260619 02:39:46.423368 8704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:39:46.423663 8704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:39:46.424813 8704 hybrid_clock.cc:648] HybridClock initialized: now 1781836786424798 us; error 32 us; skew 500 ppm
Jun 19 02:39:46 dist-test-slave-zpbg krb5kdc[8463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836786, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:39:46.427438 8704 init.cc:377] Logged in from keytab as kudu/127.5.241.66@KRBTEST.COM (short username kudu)
I20260619 02:39:46.427939 8704 webserver.cc:460] Webserver started at http://127.5.241.66:44811/ using document root <none> and password file <none>
I20260619 02:39:46.428188 8704 fs_manager.cc:359] Metadata directory not provided
I20260619 02:39:46.428262 8704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:39:46.428419 8704 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:39:46.429623 8704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-1/data/instance:
uuid: "871bed17ea3747b0a63c8a1e359d77de"
format_stamp: "Formatted at 2026-06-19 02:39:46 on dist-test-slave-zpbg"
server_key: "bc033bf5d79c5dba5f1081e15b43b518"
server_key_iv: "4e9a6d0882a2af4cd6220c82a0bf3fb6"
server_key_version: "encryptionkey@0"
I20260619 02:39:46.430006 8704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-1/wal/instance:
uuid: "871bed17ea3747b0a63c8a1e359d77de"
format_stamp: "Formatted at 2026-06-19 02:39:46 on dist-test-slave-zpbg"
server_key: "bc033bf5d79c5dba5f1081e15b43b518"
server_key_iv: "4e9a6d0882a2af4cd6220c82a0bf3fb6"
server_key_version: "encryptionkey@0"
I20260619 02:39:46.431474 8704 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:39:46.432297 8719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:46.432525 8704 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:39:46.432626 8704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-1/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-1/wal
uuid: "871bed17ea3747b0a63c8a1e359d77de"
format_stamp: "Formatted at 2026-06-19 02:39:46 on dist-test-slave-zpbg"
server_key: "bc033bf5d79c5dba5f1081e15b43b518"
server_key_iv: "4e9a6d0882a2af4cd6220c82a0bf3fb6"
server_key_version: "encryptionkey@0"
I20260619 02:39:46.432714 8704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:39:46.451933 8704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:39:46.454469 8704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:39:46.454653 8704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:39:46.454946 8704 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:39:46.455271 8704 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:39:46.455327 8704 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:46.455367 8704 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:39:46.455397 8704 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:46.461299 8704 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.66:42491
I20260619 02:39:46.461344 8832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.66:42491 every 8 connection(s)
I20260619 02:39:46.461699 8704 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
Jun 19 02:39:46 dist-test-slave-zpbg krb5kdc[8463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836786, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.66@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:39:46.469460 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 8704
I20260619 02:39:46.469561 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-1/wal/instance
I20260619 02:39:46.469761 6085 external_mini_cluster.cc:1428] Setting key 962911dffdb67790753aabcb71699f32
I20260619 02:39:46.470983 8547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:46.462508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:47421 (local address 127.5.241.126:43007)
0619 02:39:46.462562 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:46.462564 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:46.462734 (+ 170us) server_negotiation.cc:407] Connection header received
0619 02:39:46.463222 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:46.463223 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:46.463233 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:46.463297 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:46.464217 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:46.464739 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:46.465715 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:46.465897 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:46.465961 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:46.468090 (+ 2129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:46.468091 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:46.468092 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:46.468098 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:46.469417 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:46.469918 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:46.469920 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:46.469920 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:46.469983 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:46.470290 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:46.470291 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:46.470291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:46.470414 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:46.470499 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:46.470750 (+ 251us) server_negotiation.cc:299] Negotiation successful
0619 02:39:46.470846 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 02:39:46.471058 8835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:46.462410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:43007 (local address 127.5.241.66:47421)
0619 02:39:46.462575 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:46.462620 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:46.463111 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:46.463351 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:46.463354 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:46.463919 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:39:46.464114 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:46.464120 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:46.464826 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:46.464827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:46.465620 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:46.465622 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:46.465974 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:46.465975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:46.466075 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:46.466551 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:39:46.466559 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:39:46.467994 (+ 1435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:39:46.469545 (+ 1551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:46.469554 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:46.469557 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:46.469810 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:46.470123 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:46.470123 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:46.470124 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:46.470228 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:46.470489 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:39:46.470496 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:39:46.470619 (+ 123us) client_negotiation.cc:769] Sending connection context
0619 02:39:46.470705 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 02:39:46.470876 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":94,"threads_started":1}
I20260619 02:39:46.471544 8833 heartbeater.cc:347] Connected to a master server at 127.5.241.126:43007
I20260619 02:39:46.471655 8833 heartbeater.cc:464] Registering TS with master...
I20260619 02:39:46.471849 8833 heartbeater.cc:511] Master 127.5.241.126:43007 requested a full tablet report, sending...
I20260619 02:39:46.472219 8508 ts_manager.cc:194] Registered new tserver with Master: 871bed17ea3747b0a63c8a1e359d77de (127.5.241.66:42491)
I20260619 02:39:46.472584 8508 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.66@KRBTEST.COM'} at 127.5.241.66:47421
WARNING: no policy specified for kudu/127.5.241.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab.
Entry for principal kudu/127.5.241.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab.
I20260619 02:39:46.495499 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.67:0
--local_ip_for_outbound_sockets=127.5.241.67
--webserver_interface=127.5.241.67
--webserver_port=0
--tserver_master_addrs=127.5.241.126:43007
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab
--principal=kudu/127.5.241.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:38691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:39:46.574813 8838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:39:46.575070 8838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:39:46.575110 8838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:39:46.576480 8838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:39:46.576546 8838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:39:46.576593 8838 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.67
I20260619 02:39:46.578042 8838 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:38691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab
--principal=kudu/127.5.241.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.241.67
--webserver_port=0
--tserver_master_addrs=127.5.241.126:43007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8838
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.67
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:39:46.578315 8838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:39:46.578581 8838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:39:46.583758 8846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:46.583765 8843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:46.584168 8844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:39:46.584210 8838 server_base.cc:1034] running on GCE node
I20260619 02:39:46.584443 8838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:39:46.584651 8838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:39:46.585790 8838 hybrid_clock.cc:648] HybridClock initialized: now 1781836786585769 us; error 37 us; skew 500 ppm
Jun 19 02:39:46 dist-test-slave-zpbg krb5kdc[8463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836786, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:39:46.588321 8838 init.cc:377] Logged in from keytab as kudu/127.5.241.67@KRBTEST.COM (short username kudu)
I20260619 02:39:46.588861 8838 webserver.cc:460] Webserver started at http://127.5.241.67:40439/ using document root <none> and password file <none>
I20260619 02:39:46.589118 8838 fs_manager.cc:359] Metadata directory not provided
I20260619 02:39:46.589171 8838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:39:46.589350 8838 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:39:46.590461 8838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-2/data/instance:
uuid: "33095cbcf82a461bad01a59dd43cd1d9"
format_stamp: "Formatted at 2026-06-19 02:39:46 on dist-test-slave-zpbg"
server_key: "873fb97ea7af0da1248bcf8ee86e30dd"
server_key_iv: "79c521445f3e138d6d1404380ec20373"
server_key_version: "encryptionkey@0"
I20260619 02:39:46.590893 8838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-2/wal/instance:
uuid: "33095cbcf82a461bad01a59dd43cd1d9"
format_stamp: "Formatted at 2026-06-19 02:39:46 on dist-test-slave-zpbg"
server_key: "873fb97ea7af0da1248bcf8ee86e30dd"
server_key_iv: "79c521445f3e138d6d1404380ec20373"
server_key_version: "encryptionkey@0"
I20260619 02:39:46.592291 8838 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:39:46.593146 8853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:46.593353 8838 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:39:46.593459 8838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-2/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-2/wal
uuid: "33095cbcf82a461bad01a59dd43cd1d9"
format_stamp: "Formatted at 2026-06-19 02:39:46 on dist-test-slave-zpbg"
server_key: "873fb97ea7af0da1248bcf8ee86e30dd"
server_key_iv: "79c521445f3e138d6d1404380ec20373"
server_key_version: "encryptionkey@0"
I20260619 02:39:46.593549 8838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:39:46.603705 8838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:39:46.605618 8838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:39:46.605785 8838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:39:46.606012 8838 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:39:46.606317 8838 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:39:46.606371 8838 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:46.606411 8838 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:39:46.606438 8838 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:46.612986 8838 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.67:36233
I20260619 02:39:46.613037 8966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.67:36233 every 8 connection(s)
I20260619 02:39:46.613363 8838 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
Jun 19 02:39:46 dist-test-slave-zpbg krb5kdc[8463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836786, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.67@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:39:46.620357 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 8838
I20260619 02:39:46.620427 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-2/wal/instance
I20260619 02:39:46.620652 6085 external_mini_cluster.cc:1428] Setting key ad1593548d85278b0ea1e5a4c2441af7
I20260619 02:39:46.622454 8547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:46.614141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:35651 (local address 127.5.241.126:43007)
0619 02:39:46.614223 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:46.614225 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:46.614382 (+ 157us) server_negotiation.cc:407] Connection header received
0619 02:39:46.614939 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:46.614940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:46.614949 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:46.615001 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:46.615879 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:46.616363 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:46.617365 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:46.617539 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:46.617614 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:46.619802 (+ 2188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:46.619803 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:46.619803 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:46.619809 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:46.621315 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:46.621766 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:46.621766 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:46.621767 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:46.621816 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:46.622002 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:46.622002 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:46.622003 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:46.622103 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:46.622150 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:46.622336 (+ 186us) server_negotiation.cc:299] Negotiation successful
0619 02:39:46.622390 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 02:39:46.622699 8969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:46.614099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:43007 (local address 127.5.241.67:35651)
0619 02:39:46.614283 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:46.614319 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:46.614841 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:46.615055 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:46.615059 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:46.615615 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:39:46.615805 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:46.615809 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:46.616454 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:46.616455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:46.617291 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:46.617293 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:46.617631 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:46.617632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:46.617734 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:46.618299 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:39:46.618306 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:39:46.619722 (+ 1416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:39:46.621386 (+ 1664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:46.621394 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:46.621396 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:46.621700 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:46.621858 (+ 158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:46.621859 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:46.621859 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:46.621952 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:46.622141 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:39:46.622144 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:39:46.622251 (+ 107us) client_negotiation.cc:769] Sending connection context
0619 02:39:46.622326 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 02:39:46.622459 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":95,"threads_started":1}
I20260619 02:39:46.623225 8967 heartbeater.cc:347] Connected to a master server at 127.5.241.126:43007
I20260619 02:39:46.623329 8967 heartbeater.cc:464] Registering TS with master...
I20260619 02:39:46.623507 8967 heartbeater.cc:511] Master 127.5.241.126:43007 requested a full tablet report, sending...
I20260619 02:39:46.623879 8508 ts_manager.cc:194] Registered new tserver with Master: 33095cbcf82a461bad01a59dd43cd1d9 (127.5.241.67:36233)
I20260619 02:39:46.624250 8508 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.67@KRBTEST.COM'} at 127.5.241.67:35651
I20260619 02:39:46.632747 6085 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 02:39:46.643951 8547 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:39:46.643035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32930 (local address 127.5.241.126:43007)
0619 02:39:46.643095 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:46.643099 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:46.643228 (+ 129us) server_negotiation.cc:407] Connection header received
0619 02:39:46.643273 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:46.643274 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:46.643282 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:46.643345 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:46.643857 (+ 512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32930: BlockingRecv error: recv got EOF from 127.0.0.1:32930 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 02:39:46.644801 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 8570
I20260619 02:39:46.649228 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 8704
I20260619 02:39:46.653229 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 8838
I20260619 02:39:46.657111 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 8476
2026-06-19T02:39:46Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1838 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836772852275-6085-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:39:46 dist-test-slave-zpbg krb5kdc[8979](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:39:46 dist-test-slave-zpbg krb5kdc[8979](info): set up 2 sockets
Jun 19 02:39:46 dist-test-slave-zpbg krb5kdc[8979](info): commencing operation
krb5kdc: starting...
W20260619 02:39:47.785064 6085 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260619 02:39:48.107075 6085 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.322s user 0.001s sys 0.001s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260619 02:39:48.412930 6085 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.306s user 0.001s sys 0.001s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260619 02:39:48.695526 6085 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.282s user 0.001s sys 0.001s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836772852275-6085-0/krb5kdc/test-user.keytab.
W20260619 02:39:49.707648 6085 mini_kdc.cc:302] Time spent creating keytab for test-user: real 1.012s user 0.001s sys 0.001s
Jun 19 02:39:49 dist-test-slave-zpbg krb5kdc[8979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836789, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:39:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:39:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.241.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
W20260619 02:39:50.594238 6085 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.5.241.126: real 0.865s user 0.001s sys 0.002s
I20260619 02:39:50.594636 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:36255
--webserver_interface=127.5.241.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:45339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:36255
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:39:50.675935 8992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:39:50.676148 8992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:39:50.676170 8992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:39:50.677340 8992 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:39:50.677382 8992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:39:50.677397 8992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:39:50.677409 8992 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:39:50.677421 8992 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:39:50.679020 8992 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:45339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836772852275-6085-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:36255
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:36255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.241.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8992
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:39:50.679286 8992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:39:50.679544 8992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:39:50.684612 8997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:50.684577 8998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:50.684582 9000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:39:50.684618 8992 server_base.cc:1034] running on GCE node
I20260619 02:39:50.685141 8992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:39:50.685467 8992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:39:50.686668 8992 hybrid_clock.cc:648] HybridClock initialized: now 1781836790686658 us; error 38 us; skew 500 ppm
Jun 19 02:39:50 dist-test-slave-zpbg krb5kdc[8979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836790, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:39:50.692593 8992 init.cc:377] Logged in from keytab as kudu/127.5.241.126@KRBTEST.COM (short username kudu)
I20260619 02:39:50.693071 8992 webserver.cc:460] Webserver started at http://127.5.241.126:33259/ using document root <none> and password file <none>
I20260619 02:39:50.693306 8992 fs_manager.cc:359] Metadata directory not provided
I20260619 02:39:50.693375 8992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:39:50.693543 8992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:39:50.694669 8992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836772852275-6085-0/minicluster-data/master-0/data/instance:
uuid: "ddfad2ce475d49f5abdb9d3e58c4c545"
format_stamp: "Formatted at 2026-06-19 02:39:50 on dist-test-slave-zpbg"
server_key: "7eb2c3ac3dd319016c562aaf43dc6177"
server_key_iv: "46b89b12d281a63b90cef0b0b6785a11"
server_key_version: "encryptionkey@0"
I20260619 02:39:50.695115 8992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836772852275-6085-0/minicluster-data/master-0/wal/instance:
uuid: "ddfad2ce475d49f5abdb9d3e58c4c545"
format_stamp: "Formatted at 2026-06-19 02:39:50 on dist-test-slave-zpbg"
server_key: "7eb2c3ac3dd319016c562aaf43dc6177"
server_key_iv: "46b89b12d281a63b90cef0b0b6785a11"
server_key_version: "encryptionkey@0"
I20260619 02:39:50.696491 8992 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:39:50.697363 9007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:50.697566 8992 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:39:50.697697 8992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "ddfad2ce475d49f5abdb9d3e58c4c545"
format_stamp: "Formatted at 2026-06-19 02:39:50 on dist-test-slave-zpbg"
server_key: "7eb2c3ac3dd319016c562aaf43dc6177"
server_key_iv: "46b89b12d281a63b90cef0b0b6785a11"
server_key_version: "encryptionkey@0"
I20260619 02:39:50.697782 8992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:39:50.709946 8992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:39:50.712019 8992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:39:50.712199 8992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:39:50.716327 8992 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.126:36255
I20260619 02:39:50.716361 9059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.126:36255 every 8 connection(s)
I20260619 02:39:50.716717 8992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:39:50.717540 9060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:39:50.719426 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 8992
I20260619 02:39:50.719527 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836772852275-6085-0/minicluster-data/master-0/wal/instance
I20260619 02:39:50.719733 6085 external_mini_cluster.cc:1428] Setting key 5498e98617f9332b467c008569f64b5d
I20260619 02:39:50.720100 9060 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddfad2ce475d49f5abdb9d3e58c4c545: Bootstrap starting.
I20260619 02:39:50.720775 9060 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ddfad2ce475d49f5abdb9d3e58c4c545: Neither blocks nor log segments found. Creating new log.
I20260619 02:39:50.721081 9060 log.cc:824] T 00000000000000000000000000000000 P ddfad2ce475d49f5abdb9d3e58c4c545: Log is configured to *not* fsync() on all Append() calls
I20260619 02:39:50.721902 9060 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddfad2ce475d49f5abdb9d3e58c4c545: No bootstrap required, opened a new log
I20260619 02:39:50.724175 9060 raft_consensus.cc:348] T 00000000000000000000000000000000 P ddfad2ce475d49f5abdb9d3e58c4c545 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddfad2ce475d49f5abdb9d3e58c4c545" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 36255 } }
I20260619 02:39:50.724386 9060 raft_consensus.cc:374] T 00000000000000000000000000000000 P ddfad2ce475d49f5abdb9d3e58c4c545 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:39:50.724421 9060 raft_consensus.cc:729] T 00000000000000000000000000000000 P ddfad2ce475d49f5abdb9d3e58c4c545 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddfad2ce475d49f5abdb9d3e58c4c545, State: Initialized, Role: FOLLOWER
I20260619 02:39:50.724556 9060 consensus_queue.cc:260] T 00000000000000000000000000000000 P ddfad2ce475d49f5abdb9d3e58c4c545 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddfad2ce475d49f5abdb9d3e58c4c545" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 36255 } }
I20260619 02:39:50.724630 9060 raft_consensus.cc:388] T 00000000000000000000000000000000 P ddfad2ce475d49f5abdb9d3e58c4c545 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:39:50.724659 9060 raft_consensus.cc:482] T 00000000000000000000000000000000 P ddfad2ce475d49f5abdb9d3e58c4c545 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:39:50.724702 9060 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ddfad2ce475d49f5abdb9d3e58c4c545 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:39:50.725455 9060 raft_consensus.cc:504] T 00000000000000000000000000000000 P ddfad2ce475d49f5abdb9d3e58c4c545 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddfad2ce475d49f5abdb9d3e58c4c545" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 36255 } }
I20260619 02:39:50.725567 9060 leader_election.cc:302] T 00000000000000000000000000000000 P ddfad2ce475d49f5abdb9d3e58c4c545 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ddfad2ce475d49f5abdb9d3e58c4c545; no voters:
I20260619 02:39:50.725786 9060 leader_election.cc:288] T 00000000000000000000000000000000 P ddfad2ce475d49f5abdb9d3e58c4c545 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:39:50.725811 9065 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ddfad2ce475d49f5abdb9d3e58c4c545 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:39:50.725921 9065 raft_consensus.cc:686] T 00000000000000000000000000000000 P ddfad2ce475d49f5abdb9d3e58c4c545 [term 1 LEADER]: Becoming Leader. State: Replica: ddfad2ce475d49f5abdb9d3e58c4c545, State: Running, Role: LEADER
I20260619 02:39:50.726011 9065 consensus_queue.cc:237] T 00000000000000000000000000000000 P ddfad2ce475d49f5abdb9d3e58c4c545 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddfad2ce475d49f5abdb9d3e58c4c545" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 36255 } }
I20260619 02:39:50.726054 9060 sys_catalog.cc:564] T 00000000000000000000000000000000 P ddfad2ce475d49f5abdb9d3e58c4c545 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:39:50.726325 9065 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddfad2ce475d49f5abdb9d3e58c4c545 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ddfad2ce475d49f5abdb9d3e58c4c545" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddfad2ce475d49f5abdb9d3e58c4c545" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 36255 } } }
I20260619 02:39:50.726483 9065 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddfad2ce475d49f5abdb9d3e58c4c545 [sys.catalog]: This master's current role is: LEADER
I20260619 02:39:50.726671 9065 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddfad2ce475d49f5abdb9d3e58c4c545 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ddfad2ce475d49f5abdb9d3e58c4c545. Latest consensus state: current_term: 1 leader_uuid: "ddfad2ce475d49f5abdb9d3e58c4c545" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddfad2ce475d49f5abdb9d3e58c4c545" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 36255 } } }
I20260619 02:39:50.726815 9065 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddfad2ce475d49f5abdb9d3e58c4c545 [sys.catalog]: This master's current role is: LEADER
I20260619 02:39:50.727140 9072 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Jun 19 02:39:50 dist-test-slave-zpbg krb5kdc[8979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836789, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:39:50.727646 9072 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:39:50.729251 9072 catalog_manager.cc:1347] Generated new cluster ID: e2e40047d39a44e0b86cac68787b0ba5
I20260619 02:39:50.729315 9072 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:39:50.732513 9063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:50.720251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52852 (local address 127.5.241.126:36255)
0619 02:39:50.720507 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:50.720516 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:50.720551 (+ 35us) server_negotiation.cc:407] Connection header received
0619 02:39:50.721061 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:50.721069 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:50.721125 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:50.721311 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:50.722092 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:50.723486 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:50.724237 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:50.724848 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:50.724958 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:50.728040 (+ 3082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:50.728044 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:50.728044 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:50.728056 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:50.730447 (+ 2391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:50.730890 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:50.730894 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:50.730894 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:50.731000 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:50.731220 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:50.731221 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:50.731221 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:50.731528 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:50.731682 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:50.732135 (+ 453us) server_negotiation.cc:299] Negotiation successful
0619 02:39:50.732288 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":93,"threads_started":1}
I20260619 02:39:50.733410 6085 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:39:50.739789 9072 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:39:50.740566 9072 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:39:50.743870 9072 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ddfad2ce475d49f5abdb9d3e58c4c545: Generated new TSK 0
I20260619 02:39:50.744050 9072 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:39:50.751255 9063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:50.747748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52854 (local address 127.5.241.126:36255)
0619 02:39:50.747789 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:50.747792 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:50.747921 (+ 129us) server_negotiation.cc:407] Connection header received
0619 02:39:50.747996 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:50.747997 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:50.748008 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:50.748083 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:50.748470 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:50.748972 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:50.749740 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:50.749954 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:50.750031 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:50.750761 (+ 730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:50.750762 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:50.750762 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:39:50.750971 (+ 209us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:50.750987 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:39:50.750994 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:50.751060 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:50.751129 (+ 69us) server_negotiation.cc:299] Negotiation successful
0619 02:39:50.751157 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 02:39:50.753669 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 8992
2026-06-19T02:39:50Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (4093 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:39:50 dist-test-slave-zpbg krb5kdc[9093](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:39:50 dist-test-slave-zpbg krb5kdc[9093](info): set up 2 sockets
Jun 19 02:39:50 dist-test-slave-zpbg krb5kdc[9093](info): commencing operation
krb5kdc: starting...
W20260619 02:39:51.790722 6085 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Jun 19 02:39:51 dist-test-slave-zpbg krb5kdc[9093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836791, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:39:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:39:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.241.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
I20260619 02:39:51.914052 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:37479
--webserver_interface=127.5.241.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:44385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:37479
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:39:51.993908 9106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:39:51.994117 9106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:39:51.994138 9106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:39:51.995232 9106 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:39:51.995265 9106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:39:51.995280 9106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:39:51.995292 9106 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:39:51.995304 9106 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:39:51.996675 9106 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:44385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:37479
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:37479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.241.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9106
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:39:51.996870 9106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:39:51.997032 9106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:39:52.001936 9106 server_base.cc:1034] running on GCE node
W20260619 02:39:52.001910 9111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:52.001858 9114 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:52.001895 9112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:39:52.002406 9106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:39:52.002684 9106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:39:52.003840 9106 hybrid_clock.cc:648] HybridClock initialized: now 1781836792003853 us; error 70 us; skew 500 ppm
Jun 19 02:39:51 dist-test-slave-zpbg krb5kdc[9093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836791, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:39:52.006247 9106 init.cc:377] Logged in from keytab as kudu/127.5.241.126@KRBTEST.COM (short username kudu)
I20260619 02:39:52.006706 9106 webserver.cc:460] Webserver started at http://127.5.241.126:43327/ using document root <none> and password file <none>
I20260619 02:39:52.006949 9106 fs_manager.cc:359] Metadata directory not provided
I20260619 02:39:52.007018 9106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:39:52.007167 9106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:39:52.008229 9106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/master-0/data/instance:
uuid: "fc8b8f39124c457cb0a2a230c2ff3835"
format_stamp: "Formatted at 2026-06-19 02:39:52 on dist-test-slave-zpbg"
server_key: "a566ffd71a7d6a3c40f6a30c63d8f7fb"
server_key_iv: "64370a14f39e78f1d98c0d94d3d76345"
server_key_version: "encryptionkey@0"
I20260619 02:39:52.008648 9106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/master-0/wal/instance:
uuid: "fc8b8f39124c457cb0a2a230c2ff3835"
format_stamp: "Formatted at 2026-06-19 02:39:52 on dist-test-slave-zpbg"
server_key: "a566ffd71a7d6a3c40f6a30c63d8f7fb"
server_key_iv: "64370a14f39e78f1d98c0d94d3d76345"
server_key_version: "encryptionkey@0"
I20260619 02:39:52.009966 9106 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:39:52.010659 9121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:52.010860 9106 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:39:52.010980 9106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "fc8b8f39124c457cb0a2a230c2ff3835"
format_stamp: "Formatted at 2026-06-19 02:39:52 on dist-test-slave-zpbg"
server_key: "a566ffd71a7d6a3c40f6a30c63d8f7fb"
server_key_iv: "64370a14f39e78f1d98c0d94d3d76345"
server_key_version: "encryptionkey@0"
I20260619 02:39:52.011056 9106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:39:52.020149 9106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:39:52.022143 9106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:39:52.022310 9106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:39:52.026454 9106 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.126:37479
I20260619 02:39:52.026489 9173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.126:37479 every 8 connection(s)
I20260619 02:39:52.026904 9106 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:39:52.027855 9174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:39:52.028702 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 9106
I20260619 02:39:52.028815 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/master-0/wal/instance
I20260619 02:39:52.028982 6085 external_mini_cluster.cc:1428] Setting key 8f4cd5fd305740166adc892649f2ddd1
I20260619 02:39:52.030475 9174 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc8b8f39124c457cb0a2a230c2ff3835: Bootstrap starting.
I20260619 02:39:52.031219 9174 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc8b8f39124c457cb0a2a230c2ff3835: Neither blocks nor log segments found. Creating new log.
I20260619 02:39:52.031522 9174 log.cc:824] T 00000000000000000000000000000000 P fc8b8f39124c457cb0a2a230c2ff3835: Log is configured to *not* fsync() on all Append() calls
I20260619 02:39:52.032250 9174 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc8b8f39124c457cb0a2a230c2ff3835: No bootstrap required, opened a new log
I20260619 02:39:52.034700 9174 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc8b8f39124c457cb0a2a230c2ff3835 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc8b8f39124c457cb0a2a230c2ff3835" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 37479 } }
I20260619 02:39:52.034919 9174 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc8b8f39124c457cb0a2a230c2ff3835 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:39:52.034970 9174 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc8b8f39124c457cb0a2a230c2ff3835 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc8b8f39124c457cb0a2a230c2ff3835, State: Initialized, Role: FOLLOWER
I20260619 02:39:52.035131 9174 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc8b8f39124c457cb0a2a230c2ff3835 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc8b8f39124c457cb0a2a230c2ff3835" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 37479 } }
I20260619 02:39:52.035238 9174 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc8b8f39124c457cb0a2a230c2ff3835 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:39:52.035277 9174 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc8b8f39124c457cb0a2a230c2ff3835 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:39:52.035346 9174 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc8b8f39124c457cb0a2a230c2ff3835 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:39:52.036010 9174 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc8b8f39124c457cb0a2a230c2ff3835 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc8b8f39124c457cb0a2a230c2ff3835" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 37479 } }
I20260619 02:39:52.036098 9174 leader_election.cc:302] T 00000000000000000000000000000000 P fc8b8f39124c457cb0a2a230c2ff3835 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc8b8f39124c457cb0a2a230c2ff3835; no voters:
I20260619 02:39:52.036273 9174 leader_election.cc:288] T 00000000000000000000000000000000 P fc8b8f39124c457cb0a2a230c2ff3835 [CANDIDATE]: Term 1 election: Requested vote from peers
Jun 19 02:39:52 dist-test-slave-zpbg krb5kdc[9093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836791, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:39:52.036334 9179 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc8b8f39124c457cb0a2a230c2ff3835 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:39:52.036500 9179 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc8b8f39124c457cb0a2a230c2ff3835 [term 1 LEADER]: Becoming Leader. State: Replica: fc8b8f39124c457cb0a2a230c2ff3835, State: Running, Role: LEADER
I20260619 02:39:52.036536 9174 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc8b8f39124c457cb0a2a230c2ff3835 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:39:52.036669 9179 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc8b8f39124c457cb0a2a230c2ff3835 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc8b8f39124c457cb0a2a230c2ff3835" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 37479 } }
I20260619 02:39:52.037124 9180 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc8b8f39124c457cb0a2a230c2ff3835 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc8b8f39124c457cb0a2a230c2ff3835" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc8b8f39124c457cb0a2a230c2ff3835" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 37479 } } }
I20260619 02:39:52.037124 9181 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc8b8f39124c457cb0a2a230c2ff3835 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc8b8f39124c457cb0a2a230c2ff3835. Latest consensus state: current_term: 1 leader_uuid: "fc8b8f39124c457cb0a2a230c2ff3835" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc8b8f39124c457cb0a2a230c2ff3835" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 37479 } } }
I20260619 02:39:52.037262 9180 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc8b8f39124c457cb0a2a230c2ff3835 [sys.catalog]: This master's current role is: LEADER
I20260619 02:39:52.037290 9181 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc8b8f39124c457cb0a2a230c2ff3835 [sys.catalog]: This master's current role is: LEADER
I20260619 02:39:52.037775 9187 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:39:52.038556 9187 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:39:52.040863 9177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:52.029512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33120 (local address 127.5.241.126:37479)
0619 02:39:52.029680 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:52.029687 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:52.029711 (+ 24us) server_negotiation.cc:407] Connection header received
0619 02:39:52.030238 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:52.030245 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:52.030290 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:52.030406 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:52.031257 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:52.032411 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:52.033507 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:52.033900 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:52.033978 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:52.037034 (+ 3056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:52.037039 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:52.037039 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:52.037051 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:52.039220 (+ 2169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:52.039676 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:52.039680 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:52.039680 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:52.039773 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:52.040000 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:52.040001 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:52.040001 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:52.040309 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:52.040478 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:52.040560 (+ 82us) server_negotiation.cc:299] Negotiation successful
0619 02:39:52.040697 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":83,"threads_started":1}
I20260619 02:39:52.041188 9187 catalog_manager.cc:1347] Generated new cluster ID: 3fd2f313f96f42c488630be6fa7ff2e7
I20260619 02:39:52.041255 9187 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:39:52.066085 9187 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:39:52.066640 9187 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:39:52.075594 9187 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc8b8f39124c457cb0a2a230c2ff3835: Generated new TSK 0
I20260619 02:39:52.075832 9187 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.241.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab.
Entry for principal kudu/127.5.241.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab.
I20260619 02:39:52.106004 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.65:0
--local_ip_for_outbound_sockets=127.5.241.65
--webserver_interface=127.5.241.65
--webserver_port=0
--tserver_master_addrs=127.5.241.126:37479
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab
--principal=kudu/127.5.241.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:44385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:39:52.188167 9200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:39:52.188386 9200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:39:52.188410 9200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:39:52.189509 9200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:39:52.189545 9200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:39:52.189572 9200 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.65
I20260619 02:39:52.191156 9200 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:44385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab
--principal=kudu/127.5.241.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.241.65
--webserver_port=0
--tserver_master_addrs=127.5.241.126:37479
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9200
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.65
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:39:52.191347 9200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:39:52.191526 9200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:39:52.196525 9208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:39:52.196637 9200 server_base.cc:1034] running on GCE node
W20260619 02:39:52.196529 9205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:52.196583 9206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:39:52.196973 9200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:39:52.197211 9200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:39:52.198372 9200 hybrid_clock.cc:648] HybridClock initialized: now 1781836792198359 us; error 39 us; skew 500 ppm
Jun 19 02:39:52 dist-test-slave-zpbg krb5kdc[9093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836792, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:39:52.201040 9200 init.cc:377] Logged in from keytab as kudu/127.5.241.65@KRBTEST.COM (short username kudu)
I20260619 02:39:52.201531 9200 webserver.cc:460] Webserver started at http://127.5.241.65:43161/ using document root <none> and password file <none>
I20260619 02:39:52.201752 9200 fs_manager.cc:359] Metadata directory not provided
I20260619 02:39:52.201819 9200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:39:52.201977 9200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:39:52.203114 9200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-0/data/instance:
uuid: "a377294fc3164694b551c88609927385"
format_stamp: "Formatted at 2026-06-19 02:39:52 on dist-test-slave-zpbg"
server_key: "6261292af4064c8c0cea047714470290"
server_key_iv: "bbc34e7d910409cb15e22fa2490a7e1c"
server_key_version: "encryptionkey@0"
I20260619 02:39:52.203478 9200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-0/wal/instance:
uuid: "a377294fc3164694b551c88609927385"
format_stamp: "Formatted at 2026-06-19 02:39:52 on dist-test-slave-zpbg"
server_key: "6261292af4064c8c0cea047714470290"
server_key_iv: "bbc34e7d910409cb15e22fa2490a7e1c"
server_key_version: "encryptionkey@0"
I20260619 02:39:52.204854 9200 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:39:52.205648 9215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:52.205858 9200 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:39:52.205979 9200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-0/wal
uuid: "a377294fc3164694b551c88609927385"
format_stamp: "Formatted at 2026-06-19 02:39:52 on dist-test-slave-zpbg"
server_key: "6261292af4064c8c0cea047714470290"
server_key_iv: "bbc34e7d910409cb15e22fa2490a7e1c"
server_key_version: "encryptionkey@0"
I20260619 02:39:52.206053 9200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:39:52.243500 9200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:39:52.245494 9200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:39:52.245669 9200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:39:52.245926 9200 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:39:52.246238 9200 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:39:52.246291 9200 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:52.246332 9200 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:39:52.246364 9200 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:52.252184 9200 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.65:39851
I20260619 02:39:52.252231 9328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.65:39851 every 8 connection(s)
I20260619 02:39:52.252593 9200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
Jun 19 02:39:52 dist-test-slave-zpbg krb5kdc[9093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836792, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.65@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:39:52.261294 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 9200
I20260619 02:39:52.261426 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-0/wal/instance
I20260619 02:39:52.261626 6085 external_mini_cluster.cc:1428] Setting key 484b0300de2c66a626c02e5d3e6d28ba
I20260619 02:39:52.262149 9177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:52.253370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:34791 (local address 127.5.241.126:37479)
0619 02:39:52.253449 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:52.253451 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:52.253622 (+ 171us) server_negotiation.cc:407] Connection header received
0619 02:39:52.254098 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:52.254099 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:52.254109 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:52.254160 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:52.255147 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:52.255656 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:52.256672 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:52.256881 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:52.256938 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:52.259238 (+ 2300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:52.259240 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:52.259240 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:52.259247 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:52.260564 (+ 1317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:52.260982 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:52.260982 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:52.260983 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:52.261031 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:52.261341 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:52.261343 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:52.261343 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:52.261523 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:52.261616 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:52.262019 (+ 403us) server_negotiation.cc:299] Negotiation successful
0619 02:39:52.262086 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 02:39:52.262411 9331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:52.253319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:37479 (local address 127.5.241.65:34791)
0619 02:39:52.253492 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:52.253528 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:52.254013 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:52.254216 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:52.254219 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:52.254845 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:39:52.255051 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:52.255057 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:52.255768 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:52.255769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:52.256587 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:52.256590 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:52.256964 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:52.256965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:52.257072 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:52.257676 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:39:52.257685 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:39:52.259152 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:39:52.260647 (+ 1495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:52.260654 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:52.260656 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:52.260909 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:52.261119 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:52.261120 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:52.261121 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:52.261234 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:52.261605 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:39:52.261611 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:39:52.261793 (+ 182us) client_negotiation.cc:769] Sending connection context
0619 02:39:52.261934 (+ 141us) client_negotiation.cc:240] Negotiation successful
0619 02:39:52.262128 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":94,"threads_started":1}
I20260619 02:39:52.263064 9329 heartbeater.cc:347] Connected to a master server at 127.5.241.126:37479
I20260619 02:39:52.263164 9329 heartbeater.cc:464] Registering TS with master...
I20260619 02:39:52.263343 9329 heartbeater.cc:511] Master 127.5.241.126:37479 requested a full tablet report, sending...
I20260619 02:39:52.263891 9138 ts_manager.cc:194] Registered new tserver with Master: a377294fc3164694b551c88609927385 (127.5.241.65:39851)
I20260619 02:39:52.264493 9138 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.65@KRBTEST.COM'} at 127.5.241.65:34791
WARNING: no policy specified for kudu/127.5.241.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab.
Entry for principal kudu/127.5.241.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab.
I20260619 02:39:52.290452 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.66:0
--local_ip_for_outbound_sockets=127.5.241.66
--webserver_interface=127.5.241.66
--webserver_port=0
--tserver_master_addrs=127.5.241.126:37479
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab
--principal=kudu/127.5.241.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:44385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:39:52.371311 9334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:39:52.371549 9334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:39:52.371585 9334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:39:52.372707 9334 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:39:52.372773 9334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:39:52.372818 9334 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.66
I20260619 02:39:52.374253 9334 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:44385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab
--principal=kudu/127.5.241.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.241.66
--webserver_port=0
--tserver_master_addrs=127.5.241.126:37479
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9334
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.66
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:39:52.374531 9334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:39:52.374761 9334 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:39:52.379802 9342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:52.379832 9339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:52.379805 9340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:39:52.380010 9334 server_base.cc:1034] running on GCE node
I20260619 02:39:52.380219 9334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:39:52.380470 9334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:39:52.381635 9334 hybrid_clock.cc:648] HybridClock initialized: now 1781836792381619 us; error 28 us; skew 500 ppm
Jun 19 02:39:52 dist-test-slave-zpbg krb5kdc[9093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836792, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:39:52.384409 9334 init.cc:377] Logged in from keytab as kudu/127.5.241.66@KRBTEST.COM (short username kudu)
I20260619 02:39:52.384981 9334 webserver.cc:460] Webserver started at http://127.5.241.66:36711/ using document root <none> and password file <none>
I20260619 02:39:52.385246 9334 fs_manager.cc:359] Metadata directory not provided
I20260619 02:39:52.385320 9334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:39:52.385489 9334 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:39:52.386583 9334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-1/data/instance:
uuid: "61f0b447a6c1494cb1c5849afc62b0a3"
format_stamp: "Formatted at 2026-06-19 02:39:52 on dist-test-slave-zpbg"
server_key: "a2135b547ab88e3d67813fb941469b4c"
server_key_iv: "7c9bcfcc00f57e7a4cfda9062e2aeb83"
server_key_version: "encryptionkey@0"
I20260619 02:39:52.387001 9334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-1/wal/instance:
uuid: "61f0b447a6c1494cb1c5849afc62b0a3"
format_stamp: "Formatted at 2026-06-19 02:39:52 on dist-test-slave-zpbg"
server_key: "a2135b547ab88e3d67813fb941469b4c"
server_key_iv: "7c9bcfcc00f57e7a4cfda9062e2aeb83"
server_key_version: "encryptionkey@0"
I20260619 02:39:52.388497 9334 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:39:52.389335 9349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:52.389554 9334 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:39:52.389663 9334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-1/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-1/wal
uuid: "61f0b447a6c1494cb1c5849afc62b0a3"
format_stamp: "Formatted at 2026-06-19 02:39:52 on dist-test-slave-zpbg"
server_key: "a2135b547ab88e3d67813fb941469b4c"
server_key_iv: "7c9bcfcc00f57e7a4cfda9062e2aeb83"
server_key_version: "encryptionkey@0"
I20260619 02:39:52.389775 9334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:39:52.406926 9334 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:39:52.409284 9334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:39:52.409456 9334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:39:52.409696 9334 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:39:52.410071 9334 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:39:52.410125 9334 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:52.410166 9334 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:39:52.410193 9334 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:52.416399 9334 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.66:37081
I20260619 02:39:52.416463 9462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.66:37081 every 8 connection(s)
I20260619 02:39:52.416819 9334 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
Jun 19 02:39:52 dist-test-slave-zpbg krb5kdc[9093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836792, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.66@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:39:52.425624 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 9334
I20260619 02:39:52.425720 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-1/wal/instance
I20260619 02:39:52.426014 6085 external_mini_cluster.cc:1428] Setting key 8839717e5092a4174dab15936b6cb166
I20260619 02:39:52.426311 9177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:52.417636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:51525 (local address 127.5.241.126:37479)
0619 02:39:52.417698 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:52.417728 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:52.417845 (+ 117us) server_negotiation.cc:407] Connection header received
0619 02:39:52.418349 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:52.418351 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:52.418363 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:52.418422 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:52.419382 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:52.419912 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:52.420861 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:52.421068 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:52.421134 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:52.423336 (+ 2202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:52.423338 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:52.423338 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:52.423347 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:52.424819 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:52.425280 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:52.425281 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:52.425282 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:52.425343 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:52.425596 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:52.425597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:52.425597 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:52.425766 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:52.425852 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:52.426125 (+ 273us) server_negotiation.cc:299] Negotiation successful
0619 02:39:52.426210 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 02:39:52.426551 9465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:52.417567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:37479 (local address 127.5.241.66:51525)
0619 02:39:52.417721 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:52.417762 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:52.418241 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:52.418481 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:52.418484 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:52.419099 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:39:52.419288 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:52.419293 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:52.419995 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:52.419996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:52.420781 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:52.420783 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:52.421151 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:52.421152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:52.421260 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:52.421740 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:39:52.421748 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:39:52.423222 (+ 1474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:39:52.424933 (+ 1711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:52.424940 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:52.424942 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:52.425178 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:52.425413 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:52.425414 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:52.425414 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:52.425526 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:52.425843 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:39:52.425849 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:39:52.425993 (+ 144us) client_negotiation.cc:769] Sending connection context
0619 02:39:52.426105 (+ 112us) client_negotiation.cc:240] Negotiation successful
0619 02:39:52.426293 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":89,"threads_started":1}
I20260619 02:39:52.427047 9463 heartbeater.cc:347] Connected to a master server at 127.5.241.126:37479
I20260619 02:39:52.427129 9463 heartbeater.cc:464] Registering TS with master...
I20260619 02:39:52.427335 9463 heartbeater.cc:511] Master 127.5.241.126:37479 requested a full tablet report, sending...
I20260619 02:39:52.427732 9138 ts_manager.cc:194] Registered new tserver with Master: 61f0b447a6c1494cb1c5849afc62b0a3 (127.5.241.66:37081)
I20260619 02:39:52.428072 9138 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.66@KRBTEST.COM'} at 127.5.241.66:51525
WARNING: no policy specified for kudu/127.5.241.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab.
Entry for principal kudu/127.5.241.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab.
I20260619 02:39:52.455308 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.67:0
--local_ip_for_outbound_sockets=127.5.241.67
--webserver_interface=127.5.241.67
--webserver_port=0
--tserver_master_addrs=127.5.241.126:37479
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab
--principal=kudu/127.5.241.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:44385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:39:52.537037 9468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:39:52.537279 9468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:39:52.537320 9468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:39:52.538619 9468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:39:52.538686 9468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:39:52.538729 9468 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.67
I20260619 02:39:52.540174 9468 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:44385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab
--principal=kudu/127.5.241.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.241.67
--webserver_port=0
--tserver_master_addrs=127.5.241.126:37479
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9468
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.67
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:39:52.540434 9468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:39:52.540665 9468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:39:52.545840 9468 server_base.cc:1034] running on GCE node
W20260619 02:39:52.545822 9476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:52.545828 9474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:52.545811 9473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:39:52.546203 9468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:39:52.546429 9468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:39:52.547570 9468 hybrid_clock.cc:648] HybridClock initialized: now 1781836792547551 us; error 38 us; skew 500 ppm
Jun 19 02:39:52 dist-test-slave-zpbg krb5kdc[9093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836792, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:39:52.550197 9468 init.cc:377] Logged in from keytab as kudu/127.5.241.67@KRBTEST.COM (short username kudu)
I20260619 02:39:52.550666 9468 webserver.cc:460] Webserver started at http://127.5.241.67:44695/ using document root <none> and password file <none>
I20260619 02:39:52.550920 9468 fs_manager.cc:359] Metadata directory not provided
I20260619 02:39:52.550969 9468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:39:52.551100 9468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:39:52.552145 9468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-2/data/instance:
uuid: "1fb34ece1a464bb8bd7e98a7dbc0280c"
format_stamp: "Formatted at 2026-06-19 02:39:52 on dist-test-slave-zpbg"
server_key: "8b7d944bfb64d9068c1a2228212ee53e"
server_key_iv: "da771ca8b84b50ef2b82bae589a99919"
server_key_version: "encryptionkey@0"
I20260619 02:39:52.552484 9468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-2/wal/instance:
uuid: "1fb34ece1a464bb8bd7e98a7dbc0280c"
format_stamp: "Formatted at 2026-06-19 02:39:52 on dist-test-slave-zpbg"
server_key: "8b7d944bfb64d9068c1a2228212ee53e"
server_key_iv: "da771ca8b84b50ef2b82bae589a99919"
server_key_version: "encryptionkey@0"
I20260619 02:39:52.553839 9468 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:39:52.554586 9483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:52.554808 9468 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:39:52.554903 9468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-2/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-2/wal
uuid: "1fb34ece1a464bb8bd7e98a7dbc0280c"
format_stamp: "Formatted at 2026-06-19 02:39:52 on dist-test-slave-zpbg"
server_key: "8b7d944bfb64d9068c1a2228212ee53e"
server_key_iv: "da771ca8b84b50ef2b82bae589a99919"
server_key_version: "encryptionkey@0"
I20260619 02:39:52.554977 9468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:39:52.562374 9468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:39:52.564371 9468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:39:52.564538 9468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:39:52.564754 9468 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:39:52.565058 9468 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:39:52.565112 9468 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:52.565155 9468 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:39:52.565182 9468 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:52.571219 9468 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.67:40303
I20260619 02:39:52.571261 9596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.67:40303 every 8 connection(s)
I20260619 02:39:52.571575 9468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
Jun 19 02:39:52 dist-test-slave-zpbg krb5kdc[9093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836792, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.67@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:39:52.579916 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 9468
I20260619 02:39:52.579985 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836772852275-6085-0/minicluster-data/ts-2/wal/instance
I20260619 02:39:52.580186 6085 external_mini_cluster.cc:1428] Setting key a157be61d14ef32ca63008020b04cf14
I20260619 02:39:52.581424 9177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:52.572292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:54509 (local address 127.5.241.126:37479)
0619 02:39:52.572340 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:52.572342 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:52.572528 (+ 186us) server_negotiation.cc:407] Connection header received
0619 02:39:52.573061 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:52.573062 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:52.573071 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:52.573118 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:52.574133 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:52.574735 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:52.575870 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:52.576098 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:52.576182 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:52.578425 (+ 2243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:52.578427 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:52.578427 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:52.578433 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:52.579770 (+ 1337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:52.580264 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:52.580266 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:52.580266 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:52.580330 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:52.580616 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:52.580617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:52.580618 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:52.580793 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:52.580915 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:52.581286 (+ 371us) server_negotiation.cc:299] Negotiation successful
0619 02:39:52.581361 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:39:52.581521 9599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:52.572194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:37479 (local address 127.5.241.67:54509)
0619 02:39:52.572408 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:52.572454 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:52.572968 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:52.573180 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:52.573183 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:52.573770 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:39:52.573989 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:52.573995 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:52.574905 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:52.574906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:52.575784 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:52.575787 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:52.576200 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:52.576202 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:52.576353 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:52.576943 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:39:52.576952 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:39:52.578337 (+ 1385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:39:52.579871 (+ 1534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:52.579877 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:52.579879 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:52.580174 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:52.580396 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:52.580397 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:52.580398 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:52.580528 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:52.580932 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:39:52.580937 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:39:52.581088 (+ 151us) client_negotiation.cc:769] Sending connection context
0619 02:39:52.581197 (+ 109us) client_negotiation.cc:240] Negotiation successful
0619 02:39:52.581322 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":105,"threads_started":1}
I20260619 02:39:52.581971 9597 heartbeater.cc:347] Connected to a master server at 127.5.241.126:37479
I20260619 02:39:52.582077 9597 heartbeater.cc:464] Registering TS with master...
I20260619 02:39:52.582247 9597 heartbeater.cc:511] Master 127.5.241.126:37479 requested a full tablet report, sending...
I20260619 02:39:52.582625 9138 ts_manager.cc:194] Registered new tserver with Master: 1fb34ece1a464bb8bd7e98a7dbc0280c (127.5.241.67:40303)
I20260619 02:39:52.583066 9138 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.67@KRBTEST.COM'} at 127.5.241.67:54509
I20260619 02:39:52.592164 6085 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 02:39:52 dist-test-slave-zpbg krb5kdc[9093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836792, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Jun 19 02:39:52 dist-test-slave-zpbg krb5kdc[9093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836792, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:39:52.615031 9177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:52.607786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33128 (local address 127.5.241.126:37479)
0619 02:39:52.607873 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:52.607875 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:52.607940 (+ 65us) server_negotiation.cc:407] Connection header received
0619 02:39:52.607984 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:52.607985 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:52.607993 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:52.608052 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:52.608623 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:52.609105 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:52.609897 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:52.610077 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:52.610131 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:52.612338 (+ 2207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:52.612339 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:52.612339 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:52.612347 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:52.613669 (+ 1322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:52.613996 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:52.614014 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:52.614014 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:52.614078 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:52.614351 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:52.614351 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:52.614352 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:52.614531 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:52.614608 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:52.614901 (+ 293us) server_negotiation.cc:299] Negotiation successful
0619 02:39:52.614966 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260619 02:39:52.615300 9138 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:33128
I20260619 02:39:52.615830 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 9200
I20260619 02:39:52.620316 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 9334
I20260619 02:39:52.624648 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 9468
I20260619 02:39:52.629184 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 9106
2026-06-19T02:39:52Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1878 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:39:52 dist-test-slave-zpbg krb5kdc[9609](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:39:52 dist-test-slave-zpbg krb5kdc[9609](info): set up 2 sockets
Jun 19 02:39:52 dist-test-slave-zpbg krb5kdc[9609](info): commencing operation
krb5kdc: starting...
W20260619 02:39:53.674636 6085 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Jun 19 02:39:53 dist-test-slave-zpbg krb5kdc[9609](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836793, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:39:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:39:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.241.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
I20260619 02:39:53.794052 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:42987
--webserver_interface=127.5.241.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:37471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:42987
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:39:53.874717 9622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:39:53.874990 9622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:39:53.875026 9622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:39:53.876214 9622 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:39:53.876286 9622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:39:53.876309 9622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:39:53.876330 9622 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:39:53.876361 9622 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:39:53.877926 9622 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:37471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:42987
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:42987
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.241.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9622
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:39:53.878211 9622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:39:53.878448 9622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:39:53.883600 9628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:53.883733 9630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:53.883667 9627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:39:53.883666 9622 server_base.cc:1034] running on GCE node
I20260619 02:39:53.884107 9622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:39:53.884420 9622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:39:53.885612 9622 hybrid_clock.cc:648] HybridClock initialized: now 1781836793885588 us; error 36 us; skew 500 ppm
Jun 19 02:39:53 dist-test-slave-zpbg krb5kdc[9609](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836793, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:39:53.888234 9622 init.cc:377] Logged in from keytab as kudu/127.5.241.126@KRBTEST.COM (short username kudu)
I20260619 02:39:53.888707 9622 webserver.cc:460] Webserver started at http://127.5.241.126:37323/ using document root <none> and password file <none>
I20260619 02:39:53.888944 9622 fs_manager.cc:359] Metadata directory not provided
I20260619 02:39:53.888991 9622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:39:53.889119 9622 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:39:53.890193 9622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/master-0/data/instance:
uuid: "c9be757ee71a40f0a1eeb1001257b53f"
format_stamp: "Formatted at 2026-06-19 02:39:53 on dist-test-slave-zpbg"
server_key: "81a815f2c5362bb2e44a113b2d6fcaa5"
server_key_iv: "0e031254681297b5881f8f4458de7c9c"
server_key_version: "encryptionkey@0"
I20260619 02:39:53.890559 9622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/master-0/wal/instance:
uuid: "c9be757ee71a40f0a1eeb1001257b53f"
format_stamp: "Formatted at 2026-06-19 02:39:53 on dist-test-slave-zpbg"
server_key: "81a815f2c5362bb2e44a113b2d6fcaa5"
server_key_iv: "0e031254681297b5881f8f4458de7c9c"
server_key_version: "encryptionkey@0"
I20260619 02:39:53.891983 9622 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:39:53.892827 9637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:53.893026 9622 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:39:53.893144 9622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "c9be757ee71a40f0a1eeb1001257b53f"
format_stamp: "Formatted at 2026-06-19 02:39:53 on dist-test-slave-zpbg"
server_key: "81a815f2c5362bb2e44a113b2d6fcaa5"
server_key_iv: "0e031254681297b5881f8f4458de7c9c"
server_key_version: "encryptionkey@0"
I20260619 02:39:53.893216 9622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:39:53.906230 9622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:39:53.908259 9622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:39:53.908401 9622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:39:53.912405 9622 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.126:42987
I20260619 02:39:53.912436 9689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.126:42987 every 8 connection(s)
I20260619 02:39:53.912940 9622 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:39:53.913619 9690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:39:53.915961 9690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9be757ee71a40f0a1eeb1001257b53f: Bootstrap starting.
I20260619 02:39:53.916570 9690 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9be757ee71a40f0a1eeb1001257b53f: Neither blocks nor log segments found. Creating new log.
I20260619 02:39:53.916867 9690 log.cc:824] T 00000000000000000000000000000000 P c9be757ee71a40f0a1eeb1001257b53f: Log is configured to *not* fsync() on all Append() calls
I20260619 02:39:53.917570 9690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9be757ee71a40f0a1eeb1001257b53f: No bootstrap required, opened a new log
I20260619 02:39:53.919016 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 9622
I20260619 02:39:53.919107 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/master-0/wal/instance
I20260619 02:39:53.919293 6085 external_mini_cluster.cc:1428] Setting key ab823fd8ef1c0198ce603b110745e08f
I20260619 02:39:53.919358 9690 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9be757ee71a40f0a1eeb1001257b53f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9be757ee71a40f0a1eeb1001257b53f" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 42987 } }
I20260619 02:39:53.919553 9690 raft_consensus.cc:374] T 00000000000000000000000000000000 P c9be757ee71a40f0a1eeb1001257b53f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:39:53.919603 9690 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9be757ee71a40f0a1eeb1001257b53f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9be757ee71a40f0a1eeb1001257b53f, State: Initialized, Role: FOLLOWER
I20260619 02:39:53.919754 9690 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9be757ee71a40f0a1eeb1001257b53f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9be757ee71a40f0a1eeb1001257b53f" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 42987 } }
I20260619 02:39:53.919831 9690 raft_consensus.cc:388] T 00000000000000000000000000000000 P c9be757ee71a40f0a1eeb1001257b53f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:39:53.919866 9690 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9be757ee71a40f0a1eeb1001257b53f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:39:53.919904 9690 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9be757ee71a40f0a1eeb1001257b53f [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:39:53.920444 9690 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9be757ee71a40f0a1eeb1001257b53f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9be757ee71a40f0a1eeb1001257b53f" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 42987 } }
I20260619 02:39:53.920574 9690 leader_election.cc:302] T 00000000000000000000000000000000 P c9be757ee71a40f0a1eeb1001257b53f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c9be757ee71a40f0a1eeb1001257b53f; no voters:
I20260619 02:39:53.920751 9690 leader_election.cc:288] T 00000000000000000000000000000000 P c9be757ee71a40f0a1eeb1001257b53f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:39:53.920796 9695 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9be757ee71a40f0a1eeb1001257b53f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:39:53.920913 9695 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9be757ee71a40f0a1eeb1001257b53f [term 1 LEADER]: Becoming Leader. State: Replica: c9be757ee71a40f0a1eeb1001257b53f, State: Running, Role: LEADER
I20260619 02:39:53.921007 9690 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9be757ee71a40f0a1eeb1001257b53f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:39:53.921007 9695 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9be757ee71a40f0a1eeb1001257b53f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9be757ee71a40f0a1eeb1001257b53f" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 42987 } }
I20260619 02:39:53.921401 9696 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9be757ee71a40f0a1eeb1001257b53f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c9be757ee71a40f0a1eeb1001257b53f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9be757ee71a40f0a1eeb1001257b53f" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 42987 } } }
I20260619 02:39:53.921490 9696 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9be757ee71a40f0a1eeb1001257b53f [sys.catalog]: This master's current role is: LEADER
I20260619 02:39:53.921653 9697 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9be757ee71a40f0a1eeb1001257b53f [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9be757ee71a40f0a1eeb1001257b53f. Latest consensus state: current_term: 1 leader_uuid: "c9be757ee71a40f0a1eeb1001257b53f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9be757ee71a40f0a1eeb1001257b53f" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 42987 } } }
I20260619 02:39:53.921739 9697 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9be757ee71a40f0a1eeb1001257b53f [sys.catalog]: This master's current role is: LEADER
I20260619 02:39:53.921860 9702 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:39:53.922370 9702 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:39:53.924041 9702 catalog_manager.cc:1347] Generated new cluster ID: f96bf207e455423ba6d1da6fc0d7582c
I20260619 02:39:53.924098 9702 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 02:39:53 dist-test-slave-zpbg krb5kdc[9609](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836793, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:39:53.929913 9702 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:39:53.930598 9702 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:39:53.933151 9694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:53.919845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48938 (local address 127.5.241.126:42987)
0619 02:39:53.920469 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:53.920478 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:53.920503 (+ 25us) server_negotiation.cc:407] Connection header received
0619 02:39:53.920713 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:53.920722 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:53.920787 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:53.921791 (+ 1004us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:53.921918 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:53.923726 (+ 1808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:53.924825 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:53.925477 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:53.925908 (+ 431us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:53.928411 (+ 2503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:53.928415 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:53.928415 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:53.928427 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:53.931534 (+ 3107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:53.931929 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:53.931932 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:53.931932 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:53.932026 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:53.932243 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:53.932244 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:53.932245 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:53.932480 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:53.932664 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:53.932733 (+ 69us) server_negotiation.cc:299] Negotiation successful
0619 02:39:53.932993 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":536,"threads_started":1}
I20260619 02:39:53.942004 9702 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c9be757ee71a40f0a1eeb1001257b53f: Generated new TSK 0
I20260619 02:39:53.942193 9702 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.241.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab.
Entry for principal kudu/127.5.241.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab.
I20260619 02:39:53.974392 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.65:0
--local_ip_for_outbound_sockets=127.5.241.65
--webserver_interface=127.5.241.65
--webserver_port=0
--tserver_master_addrs=127.5.241.126:42987
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab
--principal=kudu/127.5.241.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:37471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:39:54.058871 9716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:39:54.059068 9716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:39:54.059092 9716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:39:54.060217 9716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:39:54.060253 9716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:39:54.060279 9716 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.65
I20260619 02:39:54.061679 9716 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:37471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab
--principal=kudu/127.5.241.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.241.65
--webserver_port=0
--tserver_master_addrs=127.5.241.126:42987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9716
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.65
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:39:54.061873 9716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:39:54.062041 9716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:39:54.066946 9721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:54.066891 9722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:54.066996 9724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:39:54.066978 9716 server_base.cc:1034] running on GCE node
I20260619 02:39:54.067309 9716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:39:54.067566 9716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:39:54.068725 9716 hybrid_clock.cc:648] HybridClock initialized: now 1781836794068706 us; error 38 us; skew 500 ppm
Jun 19 02:39:54 dist-test-slave-zpbg krb5kdc[9609](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836794, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:39:54.071444 9716 init.cc:377] Logged in from keytab as kudu/127.5.241.65@KRBTEST.COM (short username kudu)
I20260619 02:39:54.071904 9716 webserver.cc:460] Webserver started at http://127.5.241.65:38153/ using document root <none> and password file <none>
I20260619 02:39:54.072131 9716 fs_manager.cc:359] Metadata directory not provided
I20260619 02:39:54.072199 9716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:39:54.072350 9716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:39:54.073428 9716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/ts-0/data/instance:
uuid: "55bb17763b594ba1b02998baa5835630"
format_stamp: "Formatted at 2026-06-19 02:39:54 on dist-test-slave-zpbg"
server_key: "778a0fc2523b3da226deb3efb0e8df5d"
server_key_iv: "e1f4f511088355353996d23642ae5193"
server_key_version: "encryptionkey@0"
I20260619 02:39:54.073861 9716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/ts-0/wal/instance:
uuid: "55bb17763b594ba1b02998baa5835630"
format_stamp: "Formatted at 2026-06-19 02:39:54 on dist-test-slave-zpbg"
server_key: "778a0fc2523b3da226deb3efb0e8df5d"
server_key_iv: "e1f4f511088355353996d23642ae5193"
server_key_version: "encryptionkey@0"
I20260619 02:39:54.075692 9716 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260619 02:39:54.076514 9731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:54.076721 9716 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:39:54.076828 9716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/ts-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/ts-0/wal
uuid: "55bb17763b594ba1b02998baa5835630"
format_stamp: "Formatted at 2026-06-19 02:39:54 on dist-test-slave-zpbg"
server_key: "778a0fc2523b3da226deb3efb0e8df5d"
server_key_iv: "e1f4f511088355353996d23642ae5193"
server_key_version: "encryptionkey@0"
I20260619 02:39:54.076902 9716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:39:54.091605 9716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:39:54.093587 9716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:39:54.093746 9716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:39:54.093994 9716 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:39:54.094362 9716 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:39:54.094415 9716 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:54.094455 9716 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:39:54.094481 9716 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:54.100329 9716 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.65:45515
I20260619 02:39:54.100416 9844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.65:45515 every 8 connection(s)
I20260619 02:39:54.100737 9716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
Jun 19 02:39:54 dist-test-slave-zpbg krb5kdc[9609](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836794, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.65@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:39:54.109427 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 9716
I20260619 02:39:54.109543 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/ts-0/wal/instance
I20260619 02:39:54.109732 6085 external_mini_cluster.cc:1428] Setting key 5da025e8781117880cf499c59ac2f577
I20260619 02:39:54.110332 9694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:54.101478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:40973 (local address 127.5.241.126:42987)
0619 02:39:54.101547 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:54.101549 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:54.101795 (+ 246us) server_negotiation.cc:407] Connection header received
0619 02:39:54.102736 (+ 941us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:54.102737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:54.102749 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:54.102825 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:54.103455 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:54.103936 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:54.104954 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:54.105144 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:54.105193 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:54.107281 (+ 2088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:54.107282 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:54.107282 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:54.107288 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:54.108731 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:54.109137 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:54.109138 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:54.109138 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:54.109187 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:54.109576 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:54.109577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:54.109577 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:54.109719 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:54.109771 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:54.110146 (+ 375us) server_negotiation.cc:299] Negotiation successful
0619 02:39:54.110241 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 02:39:54.110499 9847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:54.101452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:42987 (local address 127.5.241.65:40973)
0619 02:39:54.101639 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:54.101681 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:54.102644 (+ 963us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:54.102911 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:54.102914 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:54.103180 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:39:54.103378 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:54.103382 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:54.104052 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:54.104053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:54.104873 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:54.104876 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:54.105215 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:54.105216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:54.105317 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:54.105778 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:39:54.105786 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:39:54.107196 (+ 1410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:39:54.108823 (+ 1627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:54.108829 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:54.108836 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:54.109067 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:54.109274 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:54.109275 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:54.109275 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:54.109448 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:54.109789 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:39:54.109796 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:39:54.109961 (+ 165us) client_negotiation.cc:769] Sending connection context
0619 02:39:54.110100 (+ 139us) client_negotiation.cc:240] Negotiation successful
0619 02:39:54.110251 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":106,"threads_started":1}
I20260619 02:39:54.110944 9845 heartbeater.cc:347] Connected to a master server at 127.5.241.126:42987
I20260619 02:39:54.111037 9845 heartbeater.cc:464] Registering TS with master...
I20260619 02:39:54.111230 9845 heartbeater.cc:511] Master 127.5.241.126:42987 requested a full tablet report, sending...
I20260619 02:39:54.111788 9654 ts_manager.cc:194] Registered new tserver with Master: 55bb17763b594ba1b02998baa5835630 (127.5.241.65:45515)
I20260619 02:39:54.112398 9654 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.65@KRBTEST.COM'} at 127.5.241.65:40973
WARNING: no policy specified for kudu/127.5.241.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab.
Entry for principal kudu/127.5.241.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab.
I20260619 02:39:54.137138 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.66:0
--local_ip_for_outbound_sockets=127.5.241.66
--webserver_interface=127.5.241.66
--webserver_port=0
--tserver_master_addrs=127.5.241.126:42987
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab
--principal=kudu/127.5.241.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:37471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:39:54.217061 9850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:39:54.217295 9850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:39:54.217331 9850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:39:54.218469 9850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:39:54.218537 9850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:39:54.218581 9850 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.66
I20260619 02:39:54.220124 9850 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:37471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab
--principal=kudu/127.5.241.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.241.66
--webserver_port=0
--tserver_master_addrs=127.5.241.126:42987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9850
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.66
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:39:54.220387 9850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:39:54.220715 9850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:39:54.225615 9855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:54.225607 9858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:54.225682 9856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:39:54.225759 9850 server_base.cc:1034] running on GCE node
I20260619 02:39:54.226118 9850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:39:54.226378 9850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:39:54.227533 9850 hybrid_clock.cc:648] HybridClock initialized: now 1781836794227527 us; error 36 us; skew 500 ppm
Jun 19 02:39:54 dist-test-slave-zpbg krb5kdc[9609](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836794, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:39:54.230018 9850 init.cc:377] Logged in from keytab as kudu/127.5.241.66@KRBTEST.COM (short username kudu)
I20260619 02:39:54.230440 9850 webserver.cc:460] Webserver started at http://127.5.241.66:43997/ using document root <none> and password file <none>
I20260619 02:39:54.230669 9850 fs_manager.cc:359] Metadata directory not provided
I20260619 02:39:54.230737 9850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:39:54.230891 9850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:39:54.231990 9850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/ts-1/data/instance:
uuid: "79ef45956f234c67b802903715b4eadd"
format_stamp: "Formatted at 2026-06-19 02:39:54 on dist-test-slave-zpbg"
server_key: "2e89c2428e4bbf657995f1afa764a2e4"
server_key_iv: "0b5c5eea4a06d7eb6b5b515cbf0aec16"
server_key_version: "encryptionkey@0"
I20260619 02:39:54.232465 9850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/ts-1/wal/instance:
uuid: "79ef45956f234c67b802903715b4eadd"
format_stamp: "Formatted at 2026-06-19 02:39:54 on dist-test-slave-zpbg"
server_key: "2e89c2428e4bbf657995f1afa764a2e4"
server_key_iv: "0b5c5eea4a06d7eb6b5b515cbf0aec16"
server_key_version: "encryptionkey@0"
I20260619 02:39:54.233870 9850 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:39:54.234609 9865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:54.234836 9850 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:39:54.234934 9850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/ts-1/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/ts-1/wal
uuid: "79ef45956f234c67b802903715b4eadd"
format_stamp: "Formatted at 2026-06-19 02:39:54 on dist-test-slave-zpbg"
server_key: "2e89c2428e4bbf657995f1afa764a2e4"
server_key_iv: "0b5c5eea4a06d7eb6b5b515cbf0aec16"
server_key_version: "encryptionkey@0"
I20260619 02:39:54.235040 9850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:39:54.242141 9850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:39:54.243976 9850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:39:54.244140 9850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:39:54.244346 9850 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:39:54.244663 9850 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:39:54.244716 9850 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:54.244748 9850 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:39:54.244776 9850 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:54.250379 9850 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.66:36685
I20260619 02:39:54.250483 9978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.66:36685 every 8 connection(s)
I20260619 02:39:54.250761 9850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
I20260619 02:39:54.251744 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 9850
I20260619 02:39:54.251852 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/ts-1/wal/instance
I20260619 02:39:54.252050 6085 external_mini_cluster.cc:1428] Setting key 04a3e868a461954f53bfdb858d4e88ce
Jun 19 02:39:54 dist-test-slave-zpbg krb5kdc[9609](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836794, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.66@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.241.67@KRBTEST.COM; defaulting to no policy
I20260619 02:39:54.261163 9694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:54.251529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:45913 (local address 127.5.241.126:42987)
0619 02:39:54.251570 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:54.251572 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:54.251760 (+ 188us) server_negotiation.cc:407] Connection header received
0619 02:39:54.252812 (+ 1052us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:54.252813 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:54.252829 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:54.252888 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:54.253506 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:54.254041 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:54.255123 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:54.255442 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:54.255524 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:54.257851 (+ 2327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:54.257853 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:54.257853 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:54.257869 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:54.259710 (+ 1841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:54.260137 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:54.260139 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:54.260139 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:54.260200 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:54.260450 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:54.260451 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:54.260452 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:54.260610 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:54.260692 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:54.261030 (+ 338us) server_negotiation.cc:299] Negotiation successful
0619 02:39:54.261099 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 02:39:54.261334 9981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:54.251474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:42987 (local address 127.5.241.66:45913)
0619 02:39:54.251638 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:54.251678 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:54.252703 (+ 1025us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:54.252956 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:54.252960 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:54.253221 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:39:54.253401 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:54.253405 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:54.254133 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:54.254135 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:54.255026 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:54.255028 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:54.255545 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:54.255546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:54.255677 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:54.256294 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:39:54.256305 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:39:54.257743 (+ 1438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:39:54.259800 (+ 2057us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:54.259806 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:54.259808 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:54.260053 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:54.260273 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:54.260273 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:54.260273 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:54.260385 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:54.260733 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:39:54.260738 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:39:54.260892 (+ 154us) client_negotiation.cc:769] Sending connection context
0619 02:39:54.260989 (+ 97us) client_negotiation.cc:240] Negotiation successful
0619 02:39:54.261111 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":102,"threads_started":1}
I20260619 02:39:54.261914 9979 heartbeater.cc:347] Connected to a master server at 127.5.241.126:42987
I20260619 02:39:54.262002 9979 heartbeater.cc:464] Registering TS with master...
I20260619 02:39:54.262147 9979 heartbeater.cc:511] Master 127.5.241.126:42987 requested a full tablet report, sending...
I20260619 02:39:54.262506 9654 ts_manager.cc:194] Registered new tserver with Master: 79ef45956f234c67b802903715b4eadd (127.5.241.66:36685)
I20260619 02:39:54.262948 9654 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.66@KRBTEST.COM'} at 127.5.241.66:45913
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab.
Entry for principal kudu/127.5.241.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab.
I20260619 02:39:54.278239 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.67:0
--local_ip_for_outbound_sockets=127.5.241.67
--webserver_interface=127.5.241.67
--webserver_port=0
--tserver_master_addrs=127.5.241.126:42987
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab
--principal=kudu/127.5.241.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:37471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:39:54.360042 9984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:39:54.360249 9984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:39:54.360273 9984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:39:54.361435 9984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:39:54.361478 9984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:39:54.361506 9984 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.67
I20260619 02:39:54.363086 9984 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:37471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab
--principal=kudu/127.5.241.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.241.67
--webserver_port=0
--tserver_master_addrs=127.5.241.126:42987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9984
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.67
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:39:54.363318 9984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:39:54.363556 9984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:39:54.368532 9992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:54.368537 9989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:54.368593 9990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:39:54.368726 9984 server_base.cc:1034] running on GCE node
I20260619 02:39:54.368947 9984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:39:54.369215 9984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:39:54.370379 9984 hybrid_clock.cc:648] HybridClock initialized: now 1781836794370371 us; error 44 us; skew 500 ppm
Jun 19 02:39:54 dist-test-slave-zpbg krb5kdc[9609](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836794, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:39:54.372874 9984 init.cc:377] Logged in from keytab as kudu/127.5.241.67@KRBTEST.COM (short username kudu)
I20260619 02:39:54.373337 9984 webserver.cc:460] Webserver started at http://127.5.241.67:42471/ using document root <none> and password file <none>
I20260619 02:39:54.373574 9984 fs_manager.cc:359] Metadata directory not provided
I20260619 02:39:54.373642 9984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:39:54.373791 9984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:39:54.374876 9984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/ts-2/data/instance:
uuid: "e9f456ef823b47ffa54711a230b111b5"
format_stamp: "Formatted at 2026-06-19 02:39:54 on dist-test-slave-zpbg"
server_key: "3e59672bc2030d438a2bc20f6f6e2c76"
server_key_iv: "02156456f8e60dcce60654211d98b4e0"
server_key_version: "encryptionkey@0"
I20260619 02:39:54.375252 9984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/ts-2/wal/instance:
uuid: "e9f456ef823b47ffa54711a230b111b5"
format_stamp: "Formatted at 2026-06-19 02:39:54 on dist-test-slave-zpbg"
server_key: "3e59672bc2030d438a2bc20f6f6e2c76"
server_key_iv: "02156456f8e60dcce60654211d98b4e0"
server_key_version: "encryptionkey@0"
I20260619 02:39:54.376564 9984 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:39:54.377367 9999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:54.377562 9984 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:39:54.377656 9984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/ts-2/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/ts-2/wal
uuid: "e9f456ef823b47ffa54711a230b111b5"
format_stamp: "Formatted at 2026-06-19 02:39:54 on dist-test-slave-zpbg"
server_key: "3e59672bc2030d438a2bc20f6f6e2c76"
server_key_iv: "02156456f8e60dcce60654211d98b4e0"
server_key_version: "encryptionkey@0"
I20260619 02:39:54.377741 9984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:39:54.389092 9984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:39:54.391280 9984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:39:54.391439 9984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:39:54.391674 9984 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:39:54.391994 9984 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:39:54.392050 9984 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:54.392092 9984 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:39:54.392119 9984 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:54.398293 9984 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.67:40075
I20260619 02:39:54.398339 10112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.67:40075 every 8 connection(s)
I20260619 02:39:54.398644 9984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
I20260619 02:39:54.403303 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 9984
I20260619 02:39:54.403398 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836772852275-6085-0/minicluster-data/ts-2/wal/instance
I20260619 02:39:54.403657 6085 external_mini_cluster.cc:1428] Setting key 14734d01e8292769a001e8254544065c
Jun 19 02:39:54 dist-test-slave-zpbg krb5kdc[9609](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836794, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.67@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:39:54.408252 9694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:54.399325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:47537 (local address 127.5.241.126:42987)
0619 02:39:54.399369 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:54.399371 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:54.399622 (+ 251us) server_negotiation.cc:407] Connection header received
0619 02:39:54.400603 (+ 981us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:54.400604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:54.400616 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:54.400669 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:54.401291 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:54.401787 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:54.402889 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:54.403085 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:54.403190 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:54.405573 (+ 2383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:54.405574 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:54.405574 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:54.405582 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:54.406856 (+ 1274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:54.407284 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:54.407286 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:54.407286 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:54.407349 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:54.407603 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:54.407603 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:54.407604 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:54.407727 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:54.407806 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:54.408076 (+ 270us) server_negotiation.cc:299] Negotiation successful
0619 02:39:54.408159 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:39:54.408459 10115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:54.399326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:42987 (local address 127.5.241.67:47537)
0619 02:39:54.399489 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:54.399534 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:54.400494 (+ 960us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:54.400745 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:54.400749 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:54.401011 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:39:54.401194 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:54.401199 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:54.401915 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:54.401916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:54.402734 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:54.402736 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:54.403211 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:54.403213 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:54.403337 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:54.403962 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:39:54.403972 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:39:54.405479 (+ 1507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:39:54.406951 (+ 1472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:54.406957 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:54.406959 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:54.407188 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:54.407417 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:54.407418 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:54.407419 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:54.407530 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:54.407795 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:39:54.407800 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:39:54.407942 (+ 142us) client_negotiation.cc:769] Sending connection context
0619 02:39:54.408054 (+ 112us) client_negotiation.cc:240] Negotiation successful
0619 02:39:54.408219 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":82,"threads_started":1}
I20260619 02:39:54.408959 10113 heartbeater.cc:347] Connected to a master server at 127.5.241.126:42987
I20260619 02:39:54.409107 10113 heartbeater.cc:464] Registering TS with master...
I20260619 02:39:54.409309 10113 heartbeater.cc:511] Master 127.5.241.126:42987 requested a full tablet report, sending...
I20260619 02:39:54.409729 9654 ts_manager.cc:194] Registered new tserver with Master: e9f456ef823b47ffa54711a230b111b5 (127.5.241.67:40075)
I20260619 02:39:54.410109 9654 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.67@KRBTEST.COM'} at 127.5.241.67:47537
I20260619 02:39:54.415602 6085 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 02:39:54 dist-test-slave-zpbg krb5kdc[9609](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836794, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 02:39:54 dist-test-slave-zpbg krb5kdc[9609](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836794, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.241.65@KRBTEST.COM
I20260619 02:39:54.439635 10120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:54.431591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38582 (local address 127.5.241.65:45515)
0619 02:39:54.431772 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:54.431778 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:54.431788 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:39:54.431815 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:54.431817 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:54.431831 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:54.431900 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:54.432532 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:54.433318 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:54.434031 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:54.434384 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:54.434442 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:54.436828 (+ 2386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:54.436841 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:54.436841 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:54.436848 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:54.438402 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:54.438758 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:54.438762 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:54.438762 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:54.438851 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:54.439078 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:54.439079 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:54.439079 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:54.439230 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:54.439365 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:54.439473 (+ 108us) server_negotiation.cc:299] Negotiation successful
0619 02:39:54.439553 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":94,"threads_started":1}
Jun 19 02:39:54 dist-test-slave-zpbg krb5kdc[9609](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836794, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:39:54.448086 9694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:54.441007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48946 (local address 127.5.241.126:42987)
0619 02:39:54.441056 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:54.441059 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:54.441070 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:39:54.441148 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:54.441149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:54.441159 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:54.441208 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:54.441734 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:54.442378 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:54.443144 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:54.443341 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:54.443425 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:54.445514 (+ 2089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:54.445515 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:54.445515 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:54.445523 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:54.446912 (+ 1389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:54.447267 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:54.447268 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:54.447269 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:54.447331 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:54.447525 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:54.447526 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:54.447527 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:54.447686 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:54.447808 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:54.447904 (+ 96us) server_negotiation.cc:299] Negotiation successful
0619 02:39:54.447984 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260619 02:39:54.448340 9654 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48946
I20260619 02:39:54.448788 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 9716
I20260619 02:39:54.453441 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 9850
I20260619 02:39:54.457784 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 9984
I20260619 02:39:54.462111 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 9622
2026-06-19T02:39:54Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1832 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:39:54 dist-test-slave-zpbg krb5kdc[10125](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:39:54 dist-test-slave-zpbg krb5kdc[10125](info): set up 2 sockets
Jun 19 02:39:54 dist-test-slave-zpbg krb5kdc[10125](info): commencing operation
krb5kdc: starting...
W20260619 02:39:55.499251 6085 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Jun 19 02:39:55 dist-test-slave-zpbg krb5kdc[10125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836795, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:39:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:39:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.241.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
I20260619 02:39:55.621951 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:41503
--webserver_interface=127.5.241.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:38817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:41503
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:39:55.710263 10138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:39:55.711233 10138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:39:55.711297 10138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:39:55.712567 10138 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:39:55.712637 10138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:39:55.712664 10138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:39:55.712694 10138 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:39:55.712724 10138 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:39:55.714335 10138 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:38817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:41503
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:41503
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.241.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10138
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:39:55.714620 10138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:39:55.714905 10138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:39:55.720046 10144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:55.720121 10146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:39:55.720253 10138 server_base.cc:1034] running on GCE node
W20260619 02:39:55.720048 10143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:39:55.720636 10138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:39:55.720898 10138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:39:55.722087 10138 hybrid_clock.cc:648] HybridClock initialized: now 1781836795722073 us; error 32 us; skew 500 ppm
Jun 19 02:39:55 dist-test-slave-zpbg krb5kdc[10125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836795, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:39:55.728408 10138 init.cc:377] Logged in from keytab as kudu/127.5.241.126@KRBTEST.COM (short username kudu)
I20260619 02:39:55.728799 10138 fs_manager.cc:359] Metadata directory not provided
I20260619 02:39:55.728876 10138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:39:55.729035 10138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:39:55.730154 10138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/master-0/data/instance:
uuid: "133ed6e1a40445918dc29bf4be4ae29b"
format_stamp: "Formatted at 2026-06-19 02:39:55 on dist-test-slave-zpbg"
server_key: "635bd63289a41f47a41cf272b209c645"
server_key_iv: "e1b6da8ce055232b6ce7473de73af1bb"
server_key_version: "encryptionkey@0"
I20260619 02:39:55.730522 10138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/master-0/wal/instance:
uuid: "133ed6e1a40445918dc29bf4be4ae29b"
format_stamp: "Formatted at 2026-06-19 02:39:55 on dist-test-slave-zpbg"
server_key: "635bd63289a41f47a41cf272b209c645"
server_key_iv: "e1b6da8ce055232b6ce7473de73af1bb"
server_key_version: "encryptionkey@0"
I20260619 02:39:55.731889 10138 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:39:55.732710 10152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:55.732960 10138 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:39:55.733067 10138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "133ed6e1a40445918dc29bf4be4ae29b"
format_stamp: "Formatted at 2026-06-19 02:39:55 on dist-test-slave-zpbg"
server_key: "635bd63289a41f47a41cf272b209c645"
server_key_iv: "e1b6da8ce055232b6ce7473de73af1bb"
server_key_version: "encryptionkey@0"
I20260619 02:39:55.733151 10138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:39:55.749163 10138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:39:55.751691 10138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:39:55.751883 10138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:39:55.755769 10138 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.126:41503
I20260619 02:39:55.755828 10204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.126:41503 every 8 connection(s)
I20260619 02:39:55.756142 10138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:39:55.757035 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 10138
I20260619 02:39:55.757105 10205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:39:55.757153 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/master-0/wal/instance
I20260619 02:39:55.757331 6085 external_mini_cluster.cc:1428] Setting key 4971fc18a38e356d8e36d8589823ec6f
I20260619 02:39:55.760528 10205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 133ed6e1a40445918dc29bf4be4ae29b: Bootstrap starting.
I20260619 02:39:55.761277 10205 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 133ed6e1a40445918dc29bf4be4ae29b: Neither blocks nor log segments found. Creating new log.
I20260619 02:39:55.761590 10205 log.cc:824] T 00000000000000000000000000000000 P 133ed6e1a40445918dc29bf4be4ae29b: Log is configured to *not* fsync() on all Append() calls
I20260619 02:39:55.762302 10205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 133ed6e1a40445918dc29bf4be4ae29b: No bootstrap required, opened a new log
Jun 19 02:39:55 dist-test-slave-zpbg krb5kdc[10125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836795, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:39:55.764380 10205 raft_consensus.cc:348] T 00000000000000000000000000000000 P 133ed6e1a40445918dc29bf4be4ae29b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "133ed6e1a40445918dc29bf4be4ae29b" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 41503 } }
I20260619 02:39:55.764555 10205 raft_consensus.cc:374] T 00000000000000000000000000000000 P 133ed6e1a40445918dc29bf4be4ae29b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:39:55.764634 10205 raft_consensus.cc:729] T 00000000000000000000000000000000 P 133ed6e1a40445918dc29bf4be4ae29b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 133ed6e1a40445918dc29bf4be4ae29b, State: Initialized, Role: FOLLOWER
I20260619 02:39:55.764772 10205 consensus_queue.cc:260] T 00000000000000000000000000000000 P 133ed6e1a40445918dc29bf4be4ae29b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "133ed6e1a40445918dc29bf4be4ae29b" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 41503 } }
I20260619 02:39:55.764856 10205 raft_consensus.cc:388] T 00000000000000000000000000000000 P 133ed6e1a40445918dc29bf4be4ae29b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:39:55.764900 10205 raft_consensus.cc:482] T 00000000000000000000000000000000 P 133ed6e1a40445918dc29bf4be4ae29b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:39:55.764951 10205 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 133ed6e1a40445918dc29bf4be4ae29b [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:39:55.768327 10208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:55.757914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40842 (local address 127.5.241.126:41503)
0619 02:39:55.758092 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:55.758101 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:55.758139 (+ 38us) server_negotiation.cc:407] Connection header received
0619 02:39:55.758627 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:55.758634 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:55.758829 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:55.758973 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:55.759821 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:55.760773 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:55.761863 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:55.762272 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:55.762414 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:55.765064 (+ 2650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:55.765068 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:55.765068 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:55.765080 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:55.766758 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:55.767199 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:55.767202 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:55.767203 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:55.767295 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:55.767512 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:55.767513 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:55.767513 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:55.767764 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:55.767910 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:55.768015 (+ 105us) server_negotiation.cc:299] Negotiation successful
0619 02:39:55.768157 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":102,"threads_started":1}
I20260619 02:39:55.787521 10205 raft_consensus.cc:504] T 00000000000000000000000000000000 P 133ed6e1a40445918dc29bf4be4ae29b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "133ed6e1a40445918dc29bf4be4ae29b" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 41503 } }
I20260619 02:39:55.787751 10205 leader_election.cc:302] T 00000000000000000000000000000000 P 133ed6e1a40445918dc29bf4be4ae29b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 133ed6e1a40445918dc29bf4be4ae29b; no voters:
I20260619 02:39:55.788020 10205 leader_election.cc:288] T 00000000000000000000000000000000 P 133ed6e1a40445918dc29bf4be4ae29b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:39:55.788112 10210 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 133ed6e1a40445918dc29bf4be4ae29b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:39:55.788311 10210 raft_consensus.cc:686] T 00000000000000000000000000000000 P 133ed6e1a40445918dc29bf4be4ae29b [term 1 LEADER]: Becoming Leader. State: Replica: 133ed6e1a40445918dc29bf4be4ae29b, State: Running, Role: LEADER
I20260619 02:39:55.788322 10205 sys_catalog.cc:564] T 00000000000000000000000000000000 P 133ed6e1a40445918dc29bf4be4ae29b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:39:55.788491 10210 consensus_queue.cc:237] T 00000000000000000000000000000000 P 133ed6e1a40445918dc29bf4be4ae29b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "133ed6e1a40445918dc29bf4be4ae29b" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 41503 } }
I20260619 02:39:55.789904 10211 sys_catalog.cc:455] T 00000000000000000000000000000000 P 133ed6e1a40445918dc29bf4be4ae29b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 133ed6e1a40445918dc29bf4be4ae29b. Latest consensus state: current_term: 1 leader_uuid: "133ed6e1a40445918dc29bf4be4ae29b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "133ed6e1a40445918dc29bf4be4ae29b" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 41503 } } }
I20260619 02:39:55.790050 10211 sys_catalog.cc:458] T 00000000000000000000000000000000 P 133ed6e1a40445918dc29bf4be4ae29b [sys.catalog]: This master's current role is: LEADER
W20260619 02:39:55.790264 10225 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 133ed6e1a40445918dc29bf4be4ae29b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:39:55.790525 10225 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:39:55.790609 10227 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:39:55.790750 10227 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:39:55.790421 10212 sys_catalog.cc:455] T 00000000000000000000000000000000 P 133ed6e1a40445918dc29bf4be4ae29b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "133ed6e1a40445918dc29bf4be4ae29b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "133ed6e1a40445918dc29bf4be4ae29b" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 41503 } } }
I20260619 02:39:55.791467 10212 sys_catalog.cc:458] T 00000000000000000000000000000000 P 133ed6e1a40445918dc29bf4be4ae29b [sys.catalog]: This master's current role is: LEADER
I20260619 02:39:55.792537 10227 catalog_manager.cc:1347] Generated new cluster ID: bf97838c39c944098ddc5b1a03975c17
I20260619 02:39:55.792595 10227 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:39:55.813220 10227 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:39:55.813839 10227 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:39:55.822872 10227 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 133ed6e1a40445918dc29bf4be4ae29b: Generated new TSK 0
I20260619 02:39:55.823128 10227 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.241.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab.
Entry for principal kudu/127.5.241.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab.
W20260619 02:39:56.359323 6085 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.5.241.65: real 0.530s user 0.000s sys 0.003s
I20260619 02:39:56.359701 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.65:0
--local_ip_for_outbound_sockets=127.5.241.65
--webserver_interface=127.5.241.65
--webserver_port=0
--tserver_master_addrs=127.5.241.126:41503
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab
--principal=kudu/127.5.241.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:38817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:39:56.441813 10231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:39:56.442065 10231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:39:56.442116 10231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:39:56.443336 10231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:39:56.443406 10231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:39:56.443454 10231 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.65
I20260619 02:39:56.445004 10231 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:38817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab
--principal=kudu/127.5.241.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.241.65
--webserver_port=0
--tserver_master_addrs=127.5.241.126:41503
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10231
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.65
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:39:56.445294 10231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:39:56.445544 10231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:39:56.450448 10231 server_base.cc:1034] running on GCE node
W20260619 02:39:56.450423 10236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:56.450464 10237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:56.450421 10239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:39:56.450933 10231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:39:56.451151 10231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:39:56.452293 10231 hybrid_clock.cc:648] HybridClock initialized: now 1781836796452271 us; error 45 us; skew 500 ppm
Jun 19 02:39:56 dist-test-slave-zpbg krb5kdc[10125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836796, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:39:56.454756 10231 init.cc:377] Logged in from keytab as kudu/127.5.241.65@KRBTEST.COM (short username kudu)
I20260619 02:39:56.455160 10231 fs_manager.cc:359] Metadata directory not provided
I20260619 02:39:56.455209 10231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:39:56.455327 10231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:39:56.456354 10231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/ts-0/data/instance:
uuid: "cee03c4eec03457cb6e7e4d0a42edce9"
format_stamp: "Formatted at 2026-06-19 02:39:56 on dist-test-slave-zpbg"
server_key: "aca8c6dacbe66b0cbf74a505fb9d3aa5"
server_key_iv: "edd25d94e10d9e45a9a4779ec30cc101"
server_key_version: "encryptionkey@0"
I20260619 02:39:56.456712 10231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/ts-0/wal/instance:
uuid: "cee03c4eec03457cb6e7e4d0a42edce9"
format_stamp: "Formatted at 2026-06-19 02:39:56 on dist-test-slave-zpbg"
server_key: "aca8c6dacbe66b0cbf74a505fb9d3aa5"
server_key_iv: "edd25d94e10d9e45a9a4779ec30cc101"
server_key_version: "encryptionkey@0"
I20260619 02:39:56.457993 10231 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:39:56.458716 10245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:56.458943 10231 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:39:56.459046 10231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/ts-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/ts-0/wal
uuid: "cee03c4eec03457cb6e7e4d0a42edce9"
format_stamp: "Formatted at 2026-06-19 02:39:56 on dist-test-slave-zpbg"
server_key: "aca8c6dacbe66b0cbf74a505fb9d3aa5"
server_key_iv: "edd25d94e10d9e45a9a4779ec30cc101"
server_key_version: "encryptionkey@0"
I20260619 02:39:56.459138 10231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:39:56.472420 10231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:39:56.474454 10231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:39:56.474619 10231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:39:56.474910 10231 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:39:56.475227 10231 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:39:56.475283 10231 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:56.475325 10231 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:39:56.475351 10231 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:56.481206 10231 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.65:39477
I20260619 02:39:56.481243 10358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.65:39477 every 8 connection(s)
I20260619 02:39:56.481526 10231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
I20260619 02:39:56.484395 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 10231
I20260619 02:39:56.484485 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/ts-0/wal/instance
I20260619 02:39:56.484688 6085 external_mini_cluster.cc:1428] Setting key 8682ecf0e1cc4126955e8f2fd1b7108f
Jun 19 02:39:56 dist-test-slave-zpbg krb5kdc[10125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836796, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.65@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:39:56.493252 10361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:56.482263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:50673 (local address 127.5.241.126:41503)
0619 02:39:56.482403 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:56.482405 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:56.482605 (+ 200us) server_negotiation.cc:407] Connection header received
0619 02:39:56.483188 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:56.483189 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:56.483201 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:56.483268 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:56.484288 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:56.484941 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:56.486465 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:56.486676 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:56.486841 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:56.489469 (+ 2628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:56.489471 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:56.489471 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:56.489480 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:56.491728 (+ 2248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:56.492156 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:56.492158 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:56.492159 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:56.492232 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:56.492482 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:56.492483 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:56.492483 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:56.492648 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:56.492723 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:56.493037 (+ 314us) server_negotiation.cc:299] Negotiation successful
0619 02:39:56.493132 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":62,"threads_started":1}
I20260619 02:39:56.493345 10362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:56.482232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:41503 (local address 127.5.241.65:50673)
0619 02:39:56.482468 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:56.482505 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:56.483075 (+ 570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:56.483345 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:56.483348 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:56.483625 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:39:56.484160 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:56.484165 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:56.485057 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:56.485058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:56.486350 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:56.486353 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:56.486826 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:56.486827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:56.486979 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:56.487563 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:39:56.487577 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:39:56.489359 (+ 1782us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:39:56.491828 (+ 2469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:56.491835 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:56.491837 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:56.492070 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:56.492305 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:56.492305 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:56.492305 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:56.492416 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:56.492740 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:39:56.492746 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:39:56.492878 (+ 132us) client_negotiation.cc:769] Sending connection context
0619 02:39:56.492991 (+ 113us) client_negotiation.cc:240] Negotiation successful
0619 02:39:56.493127 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":83,"threads_started":1}
WARNING: no policy specified for kudu/127.5.241.66@KRBTEST.COM; defaulting to no policy
I20260619 02:39:56.493918 10359 heartbeater.cc:347] Connected to a master server at 127.5.241.126:41503
I20260619 02:39:56.494019 10359 heartbeater.cc:464] Registering TS with master...
I20260619 02:39:56.494195 10359 heartbeater.cc:511] Master 127.5.241.126:41503 requested a full tablet report, sending...
I20260619 02:39:56.494712 10169 ts_manager.cc:194] Registered new tserver with Master: cee03c4eec03457cb6e7e4d0a42edce9 (127.5.241.65:39477)
I20260619 02:39:56.495456 10169 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.65@KRBTEST.COM'} at 127.5.241.65:50673
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab.
Entry for principal kudu/127.5.241.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab.
I20260619 02:39:56.513391 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.66:0
--local_ip_for_outbound_sockets=127.5.241.66
--webserver_interface=127.5.241.66
--webserver_port=0
--tserver_master_addrs=127.5.241.126:41503
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab
--principal=kudu/127.5.241.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:38817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:39:56.594888 10365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:39:56.595096 10365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:39:56.595120 10365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:39:56.596203 10365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:39:56.596237 10365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:39:56.596263 10365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.66
I20260619 02:39:56.597689 10365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:38817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab
--principal=kudu/127.5.241.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.241.66
--webserver_port=0
--tserver_master_addrs=127.5.241.126:41503
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.66
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:39:56.597868 10365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:39:56.598032 10365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:39:56.603027 10373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:56.603045 10370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:56.603034 10371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:39:56.603106 10365 server_base.cc:1034] running on GCE node
I20260619 02:39:56.603415 10365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:39:56.603653 10365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:39:56.604805 10365 hybrid_clock.cc:648] HybridClock initialized: now 1781836796604790 us; error 37 us; skew 500 ppm
Jun 19 02:39:56 dist-test-slave-zpbg krb5kdc[10125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836796, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:39:56.607352 10365 init.cc:377] Logged in from keytab as kudu/127.5.241.66@KRBTEST.COM (short username kudu)
I20260619 02:39:56.607739 10365 fs_manager.cc:359] Metadata directory not provided
I20260619 02:39:56.607810 10365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:39:56.607962 10365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:39:56.609000 10365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/ts-1/data/instance:
uuid: "361476b984224a74a778c4dbc18a0df9"
format_stamp: "Formatted at 2026-06-19 02:39:56 on dist-test-slave-zpbg"
server_key: "bcbd8ea891ecdece2d51612475a548c7"
server_key_iv: "7aa5459a98a85073f25e0f5cb49f4979"
server_key_version: "encryptionkey@0"
I20260619 02:39:56.609375 10365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/ts-1/wal/instance:
uuid: "361476b984224a74a778c4dbc18a0df9"
format_stamp: "Formatted at 2026-06-19 02:39:56 on dist-test-slave-zpbg"
server_key: "bcbd8ea891ecdece2d51612475a548c7"
server_key_iv: "7aa5459a98a85073f25e0f5cb49f4979"
server_key_version: "encryptionkey@0"
I20260619 02:39:56.610666 10365 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 02:39:56.611457 10379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:56.611624 10365 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:39:56.611730 10365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/ts-1/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/ts-1/wal
uuid: "361476b984224a74a778c4dbc18a0df9"
format_stamp: "Formatted at 2026-06-19 02:39:56 on dist-test-slave-zpbg"
server_key: "bcbd8ea891ecdece2d51612475a548c7"
server_key_iv: "7aa5459a98a85073f25e0f5cb49f4979"
server_key_version: "encryptionkey@0"
I20260619 02:39:56.611816 10365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:39:56.642212 10365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:39:56.644387 10365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:39:56.644559 10365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:39:56.644809 10365 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:39:56.645131 10365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:39:56.645187 10365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:56.645232 10365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:39:56.645277 10365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:56.651171 10365 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.66:41075
I20260619 02:39:56.651202 10492 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.66:41075 every 8 connection(s)
I20260619 02:39:56.651494 10365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
Jun 19 02:39:56 dist-test-slave-zpbg krb5kdc[10125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836796, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.66@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:39:56.658474 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 10365
I20260619 02:39:56.658622 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/ts-1/wal/instance
I20260619 02:39:56.658861 6085 external_mini_cluster.cc:1428] Setting key 9697a482bbc6f4e4077b4b0e5f8f62ed
I20260619 02:39:56.661423 10361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:56.652249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:35771 (local address 127.5.241.126:41503)
0619 02:39:56.652333 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:56.652335 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:56.652508 (+ 173us) server_negotiation.cc:407] Connection header received
0619 02:39:56.652969 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:56.652970 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:56.652979 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:56.653031 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:56.653926 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:56.654427 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:56.655496 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:56.655683 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:56.655737 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:56.658023 (+ 2286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:56.658024 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:56.658025 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:56.658030 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:56.659856 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:56.660253 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:56.660254 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:56.660255 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:56.660316 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:56.660626 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:56.660627 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:56.660627 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:56.660775 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:56.660843 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:56.661291 (+ 448us) server_negotiation.cc:299] Negotiation successful
0619 02:39:56.661361 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 02:39:56.661759 10495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:56.652208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:41503 (local address 127.5.241.66:35771)
0619 02:39:56.652390 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:56.652426 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:56.652891 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:56.653087 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:56.653090 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:56.653351 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:39:56.653849 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:56.653854 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:56.654522 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:56.654523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:56.655423 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:56.655426 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:56.655780 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:56.655782 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:56.655904 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:56.656513 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:39:56.656522 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:39:56.657930 (+ 1408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:39:56.659949 (+ 2019us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:56.659955 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:56.659957 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:56.660181 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:56.660409 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:56.660410 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:56.660410 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:56.660549 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:56.660916 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:39:56.660923 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:39:56.661113 (+ 190us) client_negotiation.cc:769] Sending connection context
0619 02:39:56.661276 (+ 163us) client_negotiation.cc:240] Negotiation successful
0619 02:39:56.661465 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":91,"threads_started":1}
I20260619 02:39:56.662431 10493 heartbeater.cc:347] Connected to a master server at 127.5.241.126:41503
I20260619 02:39:56.662540 10493 heartbeater.cc:464] Registering TS with master...
I20260619 02:39:56.662721 10493 heartbeater.cc:511] Master 127.5.241.126:41503 requested a full tablet report, sending...
I20260619 02:39:56.663146 10169 ts_manager.cc:194] Registered new tserver with Master: 361476b984224a74a778c4dbc18a0df9 (127.5.241.66:41075)
I20260619 02:39:56.663681 10169 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.66@KRBTEST.COM'} at 127.5.241.66:35771
WARNING: no policy specified for kudu/127.5.241.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab.
Entry for principal kudu/127.5.241.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab.
I20260619 02:39:56.685889 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.67:0
--local_ip_for_outbound_sockets=127.5.241.67
--webserver_interface=127.5.241.67
--webserver_port=0
--tserver_master_addrs=127.5.241.126:41503
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab
--principal=kudu/127.5.241.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:38817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:39:56.768182 10498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:39:56.768411 10498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:39:56.768450 10498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:39:56.769586 10498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:39:56.769652 10498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:39:56.769699 10498 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.67
I20260619 02:39:56.771340 10498 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:38817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab
--principal=kudu/127.5.241.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.241.67
--webserver_port=0
--tserver_master_addrs=127.5.241.126:41503
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.67
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:39:56.771598 10498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:39:56.771842 10498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:39:56.776693 10503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:56.776690 10504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:56.776690 10506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:39:56.776844 10498 server_base.cc:1034] running on GCE node
I20260619 02:39:56.777264 10498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:39:56.777519 10498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:39:56.778707 10498 hybrid_clock.cc:648] HybridClock initialized: now 1781836796778691 us; error 27 us; skew 500 ppm
Jun 19 02:39:56 dist-test-slave-zpbg krb5kdc[10125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836796, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:39:56.781387 10498 init.cc:377] Logged in from keytab as kudu/127.5.241.67@KRBTEST.COM (short username kudu)
I20260619 02:39:56.781785 10498 fs_manager.cc:359] Metadata directory not provided
I20260619 02:39:56.781858 10498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:39:56.782014 10498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:39:56.783156 10498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/ts-2/data/instance:
uuid: "f8ff2d2a79d54a7e81626a97157dd50d"
format_stamp: "Formatted at 2026-06-19 02:39:56 on dist-test-slave-zpbg"
server_key: "8f0d4b3e2915fd11c2bc2eada52fb301"
server_key_iv: "829784374fdf682e1a55adb41b474042"
server_key_version: "encryptionkey@0"
I20260619 02:39:56.783517 10498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/ts-2/wal/instance:
uuid: "f8ff2d2a79d54a7e81626a97157dd50d"
format_stamp: "Formatted at 2026-06-19 02:39:56 on dist-test-slave-zpbg"
server_key: "8f0d4b3e2915fd11c2bc2eada52fb301"
server_key_iv: "829784374fdf682e1a55adb41b474042"
server_key_version: "encryptionkey@0"
I20260619 02:39:56.784812 10498 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:39:56.785544 10512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:56.785732 10498 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:39:56.785833 10498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/ts-2/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/ts-2/wal
uuid: "f8ff2d2a79d54a7e81626a97157dd50d"
format_stamp: "Formatted at 2026-06-19 02:39:56 on dist-test-slave-zpbg"
server_key: "8f0d4b3e2915fd11c2bc2eada52fb301"
server_key_iv: "829784374fdf682e1a55adb41b474042"
server_key_version: "encryptionkey@0"
I20260619 02:39:56.785920 10498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:39:56.809501 10498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:39:56.811769 10498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:39:56.811965 10498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:39:56.812230 10498 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:39:56.812570 10498 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:39:56.812629 10498 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:56.812680 10498 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:39:56.812713 10498 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:56.819226 10498 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.67:45829
I20260619 02:39:56.819270 10625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.67:45829 every 8 connection(s)
I20260619 02:39:56.819573 10498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
I20260619 02:39:56.820905 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 10498
I20260619 02:39:56.821041 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/ts-2/wal/instance
I20260619 02:39:56.821230 6085 external_mini_cluster.cc:1428] Setting key a5276114033fd73be89604878f05992b
Jun 19 02:39:56 dist-test-slave-zpbg krb5kdc[10125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836796, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.67@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:39:56.828775 10361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:56.820281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:42237 (local address 127.5.241.126:41503)
0619 02:39:56.820334 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:56.820335 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:56.820561 (+ 226us) server_negotiation.cc:407] Connection header received
0619 02:39:56.821079 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:56.821080 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:56.821093 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:56.821142 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:56.822154 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:56.822664 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:56.823799 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:56.823996 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:56.824086 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:56.826140 (+ 2054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:56.826141 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:56.826142 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:56.826149 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:56.827426 (+ 1277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:56.827834 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:56.827835 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:56.827836 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:56.827892 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:56.828146 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:56.828147 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:56.828148 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:56.828263 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:56.828338 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:56.828606 (+ 268us) server_negotiation.cc:299] Negotiation successful
0619 02:39:56.828684 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 02:39:56.828971 10628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:56.820255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:41503 (local address 127.5.241.67:42237)
0619 02:39:56.820420 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:56.820480 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:56.820971 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:56.821212 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:56.821217 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:56.821548 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:39:56.822038 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:56.822043 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:56.822785 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:56.822787 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:56.823709 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:56.823711 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:56.824110 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:56.824112 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:56.824227 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:56.824703 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:39:56.824710 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:39:56.826028 (+ 1318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:39:56.827520 (+ 1492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:56.827527 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:56.827530 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:56.827749 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:56.827957 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:39:56.827958 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:39:56.827958 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:39:56.828062 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:39:56.828328 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:39:56.828334 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:39:56.828471 (+ 137us) client_negotiation.cc:769] Sending connection context
0619 02:39:56.828582 (+ 111us) client_negotiation.cc:240] Negotiation successful
0619 02:39:56.828737 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":73,"threads_started":1}
I20260619 02:39:56.829478 10626 heartbeater.cc:347] Connected to a master server at 127.5.241.126:41503
I20260619 02:39:56.829559 10626 heartbeater.cc:464] Registering TS with master...
I20260619 02:39:56.829679 10626 heartbeater.cc:511] Master 127.5.241.126:41503 requested a full tablet report, sending...
I20260619 02:39:56.830015 10169 ts_manager.cc:194] Registered new tserver with Master: f8ff2d2a79d54a7e81626a97157dd50d (127.5.241.67:45829)
I20260619 02:39:56.830384 10169 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.67@KRBTEST.COM'} at 127.5.241.67:42237
I20260619 02:39:56.832762 6085 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:39:56.840251 10361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:56.834026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40850 (local address 127.5.241.126:41503)
0619 02:39:56.834102 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:56.834104 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:56.834186 (+ 82us) server_negotiation.cc:407] Connection header received
0619 02:39:56.834242 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:56.834242 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:56.834250 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:56.834293 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:56.834886 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:56.835348 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:56.836211 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:56.836390 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:56.836446 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:56.837623 (+ 1177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:56.837624 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:56.837625 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:39:56.837629 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:56.838945 (+ 1316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:56.839322 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:56.839323 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:56.839323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:56.839385 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:39:56.839589 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:39:56.839590 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:39:56.839591 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:39:56.839761 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:56.839847 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:56.840028 (+ 181us) server_negotiation.cc:299] Negotiation successful
0619 02:39:56.840110 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 02:39:56.842304 10169 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40850:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:39:56.844002 10169 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:39:56.851174 10642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:56.846652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.66:41075 (local address 127.0.0.1:35884)
0619 02:39:56.846991 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:56.847002 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:56.847054 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:56.847346 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:56.847346 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:56.847348 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:39:56.847472 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:56.847474 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:56.848768 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:56.848769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:56.849951 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:56.849953 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:56.850836 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:56.850837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:56.850945 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:56.850948 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:39:56.851002 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 02:39:56.851066 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"mutex_wait_us":48,"thread_start_us":99,"threads_started":1}
I20260619 02:39:56.851192 10640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:56.846731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44056 (local address 127.5.241.65:39477)
0619 02:39:56.846943 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:56.846948 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:56.846991 (+ 43us) server_negotiation.cc:407] Connection header received
0619 02:39:56.847129 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:56.847132 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:56.847147 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:56.847224 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:39:56.847505 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:56.848666 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:56.849891 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:56.850671 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:56.850809 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:56.850880 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:56.851055 (+ 175us) server_negotiation.cc:299] Negotiation successful
0619 02:39:56.851099 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":114,"threads_started":1}
I20260619 02:39:56.851183 10639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:56.846608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:39477 (local address 127.0.0.1:44056)
0619 02:39:56.846851 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:56.846896 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:56.847015 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:56.847325 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:56.847328 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:56.847330 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:39:56.847399 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:56.847401 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:56.848785 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:56.848786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:56.849779 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:56.849781 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:56.850834 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:56.850836 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:56.850925 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:56.850942 (+ 17us) client_negotiation.cc:769] Sending connection context
0619 02:39:56.851028 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 02:39:56.851065 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"mutex_wait_us":7,"thread_start_us":87,"threads_started":1}
I20260619 02:39:56.852054 10641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:56.846812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35884 (local address 127.5.241.66:41075)
0619 02:39:56.847187 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:56.847190 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:56.847200 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:39:56.847240 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:56.847242 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:56.847258 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:56.847340 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:39:56.847561 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:56.848665 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:56.850040 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:56.850710 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:56.851828 (+ 1118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:56.851891 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:56.851919 (+ 28us) server_negotiation.cc:299] Negotiation successful
0619 02:39:56.851968 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":86,"threads_started":1}
I20260619 02:39:56.852257 10293 tablet_service.cc:1467] Processing CreateTablet for tablet d0c723d591dd4aa2b951261b5d77b69a (DEFAULT_TABLE table=test-table [id=c71f6b70c117404d870d0ad21785d79b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:39:56.852449 10638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:56.846866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.67:45829 (local address 127.0.0.1:49972)
0619 02:39:56.847344 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:56.847355 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:56.847402 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:56.847675 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:56.847675 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:56.847677 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:39:56.847721 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:56.847722 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:56.848582 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:56.848583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:56.849951 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:56.849953 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:56.852203 (+ 2250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:56.852205 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:56.852264 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:56.852268 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:39:56.852327 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 02:39:56.852370 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"mutex_wait_us":109,"thread_start_us":57,"threads_started":1}
I20260619 02:39:56.852464 10643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:56.847044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49972 (local address 127.5.241.67:45829)
0619 02:39:56.847235 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:56.847239 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:56.847458 (+ 219us) server_negotiation.cc:407] Connection header received
0619 02:39:56.847503 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:56.847505 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:56.847520 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:56.847589 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:39:56.847798 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:56.848473 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:56.850825 (+ 2352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:56.851654 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:56.851719 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:56.851775 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:56.852343 (+ 568us) server_negotiation.cc:299] Negotiation successful
0619 02:39:56.852384 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":103,"threads_started":1}
I20260619 02:39:56.852631 10293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0c723d591dd4aa2b951261b5d77b69a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:39:56.852698 10427 tablet_service.cc:1467] Processing CreateTablet for tablet d0c723d591dd4aa2b951261b5d77b69a (DEFAULT_TABLE table=test-table [id=c71f6b70c117404d870d0ad21785d79b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:39:56.853014 10427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0c723d591dd4aa2b951261b5d77b69a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:39:56.855568 10560 tablet_service.cc:1467] Processing CreateTablet for tablet d0c723d591dd4aa2b951261b5d77b69a (DEFAULT_TABLE table=test-table [id=c71f6b70c117404d870d0ad21785d79b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:39:56.855902 10560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0c723d591dd4aa2b951261b5d77b69a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:39:56.856159 10645 tablet_bootstrap.cc:492] T d0c723d591dd4aa2b951261b5d77b69a P 361476b984224a74a778c4dbc18a0df9: Bootstrap starting.
I20260619 02:39:56.856444 10644 tablet_bootstrap.cc:492] T d0c723d591dd4aa2b951261b5d77b69a P cee03c4eec03457cb6e7e4d0a42edce9: Bootstrap starting.
I20260619 02:39:56.856864 10645 tablet_bootstrap.cc:654] T d0c723d591dd4aa2b951261b5d77b69a P 361476b984224a74a778c4dbc18a0df9: Neither blocks nor log segments found. Creating new log.
I20260619 02:39:56.857026 10644 tablet_bootstrap.cc:654] T d0c723d591dd4aa2b951261b5d77b69a P cee03c4eec03457cb6e7e4d0a42edce9: Neither blocks nor log segments found. Creating new log.
I20260619 02:39:56.857491 10645 log.cc:824] T d0c723d591dd4aa2b951261b5d77b69a P 361476b984224a74a778c4dbc18a0df9: Log is configured to *not* fsync() on all Append() calls
I20260619 02:39:56.857838 10644 log.cc:824] T d0c723d591dd4aa2b951261b5d77b69a P cee03c4eec03457cb6e7e4d0a42edce9: Log is configured to *not* fsync() on all Append() calls
I20260619 02:39:56.858372 10645 tablet_bootstrap.cc:492] T d0c723d591dd4aa2b951261b5d77b69a P 361476b984224a74a778c4dbc18a0df9: No bootstrap required, opened a new log
I20260619 02:39:56.858520 10645 ts_tablet_manager.cc:1397] T d0c723d591dd4aa2b951261b5d77b69a P 361476b984224a74a778c4dbc18a0df9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:39:56.858685 10644 tablet_bootstrap.cc:492] T d0c723d591dd4aa2b951261b5d77b69a P cee03c4eec03457cb6e7e4d0a42edce9: No bootstrap required, opened a new log
I20260619 02:39:56.858816 10644 ts_tablet_manager.cc:1397] T d0c723d591dd4aa2b951261b5d77b69a P cee03c4eec03457cb6e7e4d0a42edce9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:39:56.859584 10648 tablet_bootstrap.cc:492] T d0c723d591dd4aa2b951261b5d77b69a P f8ff2d2a79d54a7e81626a97157dd50d: Bootstrap starting.
I20260619 02:39:56.860479 10648 tablet_bootstrap.cc:654] T d0c723d591dd4aa2b951261b5d77b69a P f8ff2d2a79d54a7e81626a97157dd50d: Neither blocks nor log segments found. Creating new log.
I20260619 02:39:56.860561 10645 raft_consensus.cc:348] T d0c723d591dd4aa2b951261b5d77b69a P 361476b984224a74a778c4dbc18a0df9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cee03c4eec03457cb6e7e4d0a42edce9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 39477 } } peers { permanent_uuid: "361476b984224a74a778c4dbc18a0df9" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 41075 } } peers { permanent_uuid: "f8ff2d2a79d54a7e81626a97157dd50d" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 45829 } }
I20260619 02:39:56.860749 10645 raft_consensus.cc:374] T d0c723d591dd4aa2b951261b5d77b69a P 361476b984224a74a778c4dbc18a0df9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:39:56.860811 10645 raft_consensus.cc:729] T d0c723d591dd4aa2b951261b5d77b69a P 361476b984224a74a778c4dbc18a0df9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 361476b984224a74a778c4dbc18a0df9, State: Initialized, Role: FOLLOWER
I20260619 02:39:56.860950 10645 consensus_queue.cc:260] T d0c723d591dd4aa2b951261b5d77b69a P 361476b984224a74a778c4dbc18a0df9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cee03c4eec03457cb6e7e4d0a42edce9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 39477 } } peers { permanent_uuid: "361476b984224a74a778c4dbc18a0df9" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 41075 } } peers { permanent_uuid: "f8ff2d2a79d54a7e81626a97157dd50d" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 45829 } }
I20260619 02:39:56.861079 10648 log.cc:824] T d0c723d591dd4aa2b951261b5d77b69a P f8ff2d2a79d54a7e81626a97157dd50d: Log is configured to *not* fsync() on all Append() calls
I20260619 02:39:56.861090 10644 raft_consensus.cc:348] T d0c723d591dd4aa2b951261b5d77b69a P cee03c4eec03457cb6e7e4d0a42edce9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cee03c4eec03457cb6e7e4d0a42edce9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 39477 } } peers { permanent_uuid: "361476b984224a74a778c4dbc18a0df9" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 41075 } } peers { permanent_uuid: "f8ff2d2a79d54a7e81626a97157dd50d" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 45829 } }
I20260619 02:39:56.861268 10644 raft_consensus.cc:374] T d0c723d591dd4aa2b951261b5d77b69a P cee03c4eec03457cb6e7e4d0a42edce9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:39:56.861338 10644 raft_consensus.cc:729] T d0c723d591dd4aa2b951261b5d77b69a P cee03c4eec03457cb6e7e4d0a42edce9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cee03c4eec03457cb6e7e4d0a42edce9, State: Initialized, Role: FOLLOWER
I20260619 02:39:56.861369 10645 ts_tablet_manager.cc:1428] T d0c723d591dd4aa2b951261b5d77b69a P 361476b984224a74a778c4dbc18a0df9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 02:39:56.861488 10644 consensus_queue.cc:260] T d0c723d591dd4aa2b951261b5d77b69a P cee03c4eec03457cb6e7e4d0a42edce9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cee03c4eec03457cb6e7e4d0a42edce9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 39477 } } peers { permanent_uuid: "361476b984224a74a778c4dbc18a0df9" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 41075 } } peers { permanent_uuid: "f8ff2d2a79d54a7e81626a97157dd50d" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 45829 } }
I20260619 02:39:56.861555 10493 heartbeater.cc:503] Master 127.5.241.126:41503 was elected leader, sending a full tablet report...
I20260619 02:39:56.861802 10644 ts_tablet_manager.cc:1428] T d0c723d591dd4aa2b951261b5d77b69a P cee03c4eec03457cb6e7e4d0a42edce9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 02:39:56.861811 10359 heartbeater.cc:503] Master 127.5.241.126:41503 was elected leader, sending a full tablet report...
I20260619 02:39:56.862162 10648 tablet_bootstrap.cc:492] T d0c723d591dd4aa2b951261b5d77b69a P f8ff2d2a79d54a7e81626a97157dd50d: No bootstrap required, opened a new log
I20260619 02:39:56.862306 10648 ts_tablet_manager.cc:1397] T d0c723d591dd4aa2b951261b5d77b69a P f8ff2d2a79d54a7e81626a97157dd50d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 02:39:56.864197 10648 raft_consensus.cc:348] T d0c723d591dd4aa2b951261b5d77b69a P f8ff2d2a79d54a7e81626a97157dd50d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cee03c4eec03457cb6e7e4d0a42edce9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 39477 } } peers { permanent_uuid: "361476b984224a74a778c4dbc18a0df9" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 41075 } } peers { permanent_uuid: "f8ff2d2a79d54a7e81626a97157dd50d" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 45829 } }
I20260619 02:39:56.864346 10648 raft_consensus.cc:374] T d0c723d591dd4aa2b951261b5d77b69a P f8ff2d2a79d54a7e81626a97157dd50d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:39:56.864374 10648 raft_consensus.cc:729] T d0c723d591dd4aa2b951261b5d77b69a P f8ff2d2a79d54a7e81626a97157dd50d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8ff2d2a79d54a7e81626a97157dd50d, State: Initialized, Role: FOLLOWER
I20260619 02:39:56.864495 10648 consensus_queue.cc:260] T d0c723d591dd4aa2b951261b5d77b69a P f8ff2d2a79d54a7e81626a97157dd50d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cee03c4eec03457cb6e7e4d0a42edce9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 39477 } } peers { permanent_uuid: "361476b984224a74a778c4dbc18a0df9" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 41075 } } peers { permanent_uuid: "f8ff2d2a79d54a7e81626a97157dd50d" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 45829 } }
I20260619 02:39:56.864763 10648 ts_tablet_manager.cc:1428] T d0c723d591dd4aa2b951261b5d77b69a P f8ff2d2a79d54a7e81626a97157dd50d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 02:39:56.864876 10626 heartbeater.cc:503] Master 127.5.241.126:41503 was elected leader, sending a full tablet report...
I20260619 02:39:56.875854 10652 raft_consensus.cc:482] T d0c723d591dd4aa2b951261b5d77b69a P f8ff2d2a79d54a7e81626a97157dd50d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:39:56.875985 10652 raft_consensus.cc:504] T d0c723d591dd4aa2b951261b5d77b69a P f8ff2d2a79d54a7e81626a97157dd50d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cee03c4eec03457cb6e7e4d0a42edce9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 39477 } } peers { permanent_uuid: "361476b984224a74a778c4dbc18a0df9" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 41075 } } peers { permanent_uuid: "f8ff2d2a79d54a7e81626a97157dd50d" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 45829 } }
I20260619 02:39:56.876291 10652 leader_election.cc:288] T d0c723d591dd4aa2b951261b5d77b69a P f8ff2d2a79d54a7e81626a97157dd50d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cee03c4eec03457cb6e7e4d0a42edce9 (127.5.241.65:39477), 361476b984224a74a778c4dbc18a0df9 (127.5.241.66:41075)
I20260619 02:39:56.879233 10628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:56.876526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:39477 (local address 127.5.241.67:60819)
0619 02:39:56.876601 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:56.876617 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:56.876657 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:56.876834 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:56.876835 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:56.876836 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:39:56.876896 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:56.876897 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:56.877566 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:56.877567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:56.878322 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:56.878324 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:56.878958 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:56.878959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:56.879042 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:56.879045 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:39:56.879120 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 02:39:56.879160 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260619 02:39:56.879233 10640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:56.876640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:60819 (local address 127.5.241.65:39477)
0619 02:39:56.876700 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:56.876703 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:56.876712 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:39:56.876742 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:56.876743 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:56.876753 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:56.876827 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:39:56.876974 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:56.877479 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:56.878432 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:56.878871 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:56.878959 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:56.878969 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:56.879130 (+ 161us) server_negotiation.cc:299] Negotiation successful
0619 02:39:56.879166 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 02:39:56.879383 10641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:56.876633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:50269 (local address 127.5.241.66:41075)
0619 02:39:56.876679 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:56.876681 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:56.876821 (+ 140us) server_negotiation.cc:407] Connection header received
0619 02:39:56.876955 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:56.876957 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:56.876966 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:56.877015 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:39:56.877173 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:56.877632 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:56.878568 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:56.879034 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:56.879135 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:56.879144 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:56.879289 (+ 145us) server_negotiation.cc:299] Negotiation successful
0619 02:39:56.879323 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 02:39:56.879654 10653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:56.876526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.66:41075 (local address 127.5.241.67:50269)
0619 02:39:56.876761 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:56.876772 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:56.876845 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:56.877016 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:56.877017 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:56.877018 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:39:56.877089 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:56.877096 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:56.877723 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:56.877724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:56.878478 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:56.878480 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:56.879137 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:56.879138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:56.879213 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:56.879217 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:39:56.879283 (+ 66us) client_negotiation.cc:240] Negotiation successful
0619 02:39:56.879321 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":13,"thread_start_us":124,"threads_started":1}
I20260619 02:39:56.879657 10313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0c723d591dd4aa2b951261b5d77b69a" candidate_uuid: "f8ff2d2a79d54a7e81626a97157dd50d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cee03c4eec03457cb6e7e4d0a42edce9" is_pre_election: true
I20260619 02:39:56.879808 10313 raft_consensus.cc:2454] T d0c723d591dd4aa2b951261b5d77b69a P cee03c4eec03457cb6e7e4d0a42edce9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8ff2d2a79d54a7e81626a97157dd50d in term 0.
I20260619 02:39:56.880019 10515 leader_election.cc:302] T d0c723d591dd4aa2b951261b5d77b69a P f8ff2d2a79d54a7e81626a97157dd50d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cee03c4eec03457cb6e7e4d0a42edce9, f8ff2d2a79d54a7e81626a97157dd50d; no voters:
I20260619 02:39:56.880038 10447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0c723d591dd4aa2b951261b5d77b69a" candidate_uuid: "f8ff2d2a79d54a7e81626a97157dd50d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "361476b984224a74a778c4dbc18a0df9" is_pre_election: true
I20260619 02:39:56.880153 10652 raft_consensus.cc:2781] T d0c723d591dd4aa2b951261b5d77b69a P f8ff2d2a79d54a7e81626a97157dd50d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:39:56.880187 10447 raft_consensus.cc:2454] T d0c723d591dd4aa2b951261b5d77b69a P 361476b984224a74a778c4dbc18a0df9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8ff2d2a79d54a7e81626a97157dd50d in term 0.
I20260619 02:39:56.880201 10652 raft_consensus.cc:482] T d0c723d591dd4aa2b951261b5d77b69a P f8ff2d2a79d54a7e81626a97157dd50d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:39:56.880230 10652 raft_consensus.cc:3037] T d0c723d591dd4aa2b951261b5d77b69a P f8ff2d2a79d54a7e81626a97157dd50d [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:39:56.880939 10652 raft_consensus.cc:504] T d0c723d591dd4aa2b951261b5d77b69a P f8ff2d2a79d54a7e81626a97157dd50d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cee03c4eec03457cb6e7e4d0a42edce9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 39477 } } peers { permanent_uuid: "361476b984224a74a778c4dbc18a0df9" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 41075 } } peers { permanent_uuid: "f8ff2d2a79d54a7e81626a97157dd50d" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 45829 } }
I20260619 02:39:56.881083 10652 leader_election.cc:288] T d0c723d591dd4aa2b951261b5d77b69a P f8ff2d2a79d54a7e81626a97157dd50d [CANDIDATE]: Term 1 election: Requested vote from peers cee03c4eec03457cb6e7e4d0a42edce9 (127.5.241.65:39477), 361476b984224a74a778c4dbc18a0df9 (127.5.241.66:41075)
I20260619 02:39:56.881223 10313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0c723d591dd4aa2b951261b5d77b69a" candidate_uuid: "f8ff2d2a79d54a7e81626a97157dd50d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cee03c4eec03457cb6e7e4d0a42edce9"
I20260619 02:39:56.881249 10447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0c723d591dd4aa2b951261b5d77b69a" candidate_uuid: "f8ff2d2a79d54a7e81626a97157dd50d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "361476b984224a74a778c4dbc18a0df9"
I20260619 02:39:56.881299 10313 raft_consensus.cc:3037] T d0c723d591dd4aa2b951261b5d77b69a P cee03c4eec03457cb6e7e4d0a42edce9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:39:56.881347 10447 raft_consensus.cc:3037] T d0c723d591dd4aa2b951261b5d77b69a P 361476b984224a74a778c4dbc18a0df9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:39:56.882078 10313 raft_consensus.cc:2454] T d0c723d591dd4aa2b951261b5d77b69a P cee03c4eec03457cb6e7e4d0a42edce9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8ff2d2a79d54a7e81626a97157dd50d in term 1.
I20260619 02:39:56.882217 10447 raft_consensus.cc:2454] T d0c723d591dd4aa2b951261b5d77b69a P 361476b984224a74a778c4dbc18a0df9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8ff2d2a79d54a7e81626a97157dd50d in term 1.
I20260619 02:39:56.882232 10515 leader_election.cc:302] T d0c723d591dd4aa2b951261b5d77b69a P f8ff2d2a79d54a7e81626a97157dd50d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cee03c4eec03457cb6e7e4d0a42edce9, f8ff2d2a79d54a7e81626a97157dd50d; no voters:
I20260619 02:39:56.882337 10652 raft_consensus.cc:2781] T d0c723d591dd4aa2b951261b5d77b69a P f8ff2d2a79d54a7e81626a97157dd50d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:39:56.882532 10652 raft_consensus.cc:686] T d0c723d591dd4aa2b951261b5d77b69a P f8ff2d2a79d54a7e81626a97157dd50d [term 1 LEADER]: Becoming Leader. State: Replica: f8ff2d2a79d54a7e81626a97157dd50d, State: Running, Role: LEADER
I20260619 02:39:56.882645 10652 consensus_queue.cc:237] T d0c723d591dd4aa2b951261b5d77b69a P f8ff2d2a79d54a7e81626a97157dd50d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cee03c4eec03457cb6e7e4d0a42edce9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 39477 } } peers { permanent_uuid: "361476b984224a74a778c4dbc18a0df9" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 41075 } } peers { permanent_uuid: "f8ff2d2a79d54a7e81626a97157dd50d" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 45829 } }
I20260619 02:39:56.883472 10169 catalog_manager.cc:5495] T d0c723d591dd4aa2b951261b5d77b69a P f8ff2d2a79d54a7e81626a97157dd50d reported cstate change: term changed from 0 to 1, leader changed from <none> to f8ff2d2a79d54a7e81626a97157dd50d (127.5.241.67). New cstate: current_term: 1 leader_uuid: "f8ff2d2a79d54a7e81626a97157dd50d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cee03c4eec03457cb6e7e4d0a42edce9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 39477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "361476b984224a74a778c4dbc18a0df9" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f8ff2d2a79d54a7e81626a97157dd50d" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 45829 } health_report { overall_health: HEALTHY } } }
I20260619 02:39:56.896692 10643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:56.894418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49974 (local address 127.5.241.67:45829)
0619 02:39:56.894479 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:56.894483 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:56.894496 (+ 13us) server_negotiation.cc:407] Connection header received
0619 02:39:56.894526 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:56.894527 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:56.894543 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:56.894616 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:39:56.894763 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:56.895318 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:56.896040 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:56.896218 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:56.896271 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:56.896407 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:39:56.896502 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:39:56.896562 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:56.896599 (+ 37us) server_negotiation.cc:299] Negotiation successful
0619 02:39:56.896616 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 02:39:56.898522 10447 raft_consensus.cc:1261] T d0c723d591dd4aa2b951261b5d77b69a P 361476b984224a74a778c4dbc18a0df9 [term 1 FOLLOWER]: Refusing update from remote peer f8ff2d2a79d54a7e81626a97157dd50d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:39:56.898524 10313 raft_consensus.cc:1261] T d0c723d591dd4aa2b951261b5d77b69a P cee03c4eec03457cb6e7e4d0a42edce9 [term 1 FOLLOWER]: Refusing update from remote peer f8ff2d2a79d54a7e81626a97157dd50d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:39:56.898984 10654 consensus_queue.cc:1035] T d0c723d591dd4aa2b951261b5d77b69a P f8ff2d2a79d54a7e81626a97157dd50d [LEADER]: Connected to new peer: Peer: permanent_uuid: "361476b984224a74a778c4dbc18a0df9" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:39:56.899127 10652 consensus_queue.cc:1035] T d0c723d591dd4aa2b951261b5d77b69a P f8ff2d2a79d54a7e81626a97157dd50d [LEADER]: Connected to new peer: Peer: permanent_uuid: "cee03c4eec03457cb6e7e4d0a42edce9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 39477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:39:56.901932 10657 mvcc.cc:205] Tried to move back new op lower bound from 7298403520093536256 to 7298403520032067584. Current Snapshot: MvccSnapshot[applied={T|T < 7298403520093536256}]
I20260619 02:39:56.902912 10659 mvcc.cc:205] Tried to move back new op lower bound from 7298403520093536256 to 7298403520032067584. Current Snapshot: MvccSnapshot[applied={T|T < 7298403520093536256}]
I20260619 02:39:56.904304 10168 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:39:56.904410 10168 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:39:56.905454 10168 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 133ed6e1a40445918dc29bf4be4ae29b: Sending DeleteTablet for 3 replicas of tablet d0c723d591dd4aa2b951261b5d77b69a
I20260619 02:39:56.905819 10293 tablet_service.cc:1514] Processing DeleteTablet for tablet d0c723d591dd4aa2b951261b5d77b69a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:39:56 UTC) from {username='kudu', principal='kudu/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:44056
I20260619 02:39:56.905844 10560 tablet_service.cc:1514] Processing DeleteTablet for tablet d0c723d591dd4aa2b951261b5d77b69a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:39:56 UTC) from {username='kudu', principal='kudu/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:49972
I20260619 02:39:56.905936 10427 tablet_service.cc:1514] Processing DeleteTablet for tablet d0c723d591dd4aa2b951261b5d77b69a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:39:56 UTC) from {username='kudu', principal='kudu/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:35884
I20260619 02:39:56.906065 10666 tablet_replica.cc:330] T d0c723d591dd4aa2b951261b5d77b69a P f8ff2d2a79d54a7e81626a97157dd50d: stopping tablet replica
I20260619 02:39:56.906184 10666 raft_consensus.cc:2229] T d0c723d591dd4aa2b951261b5d77b69a P f8ff2d2a79d54a7e81626a97157dd50d [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:39:56.906288 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 10231
I20260619 02:39:56.906313 10666 raft_consensus.cc:2258] T d0c723d591dd4aa2b951261b5d77b69a P f8ff2d2a79d54a7e81626a97157dd50d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:39:56.907994 10666 ts_tablet_manager.cc:1905] T d0c723d591dd4aa2b951261b5d77b69a P f8ff2d2a79d54a7e81626a97157dd50d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:39:56.909981 10666 ts_tablet_manager.cc:1918] T d0c723d591dd4aa2b951261b5d77b69a P f8ff2d2a79d54a7e81626a97157dd50d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:39:56.910061 10666 log.cc:1196] T d0c723d591dd4aa2b951261b5d77b69a P f8ff2d2a79d54a7e81626a97157dd50d: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836772852275-6085-0/minicluster-data/ts-2/wal/wals/d0c723d591dd4aa2b951261b5d77b69a
I20260619 02:39:56.910357 10666 ts_tablet_manager.cc:1939] T d0c723d591dd4aa2b951261b5d77b69a P f8ff2d2a79d54a7e81626a97157dd50d: Deleting consensus metadata
I20260619 02:39:56.910820 10155 catalog_manager.cc:4841] TS f8ff2d2a79d54a7e81626a97157dd50d (127.5.241.67:45829): tablet d0c723d591dd4aa2b951261b5d77b69a (table test-table [id=c71f6b70c117404d870d0ad21785d79b]) successfully deleted
W20260619 02:39:56.912071 10155 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.241.65:39477 (error 108)
I20260619 02:39:56.912261 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 10365
I20260619 02:39:56.913347 10638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:56.913189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:39477 (local address 127.0.0.1:44064)
0619 02:39:56.913260 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:56.913310 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.65:39477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260619 02:39:56.913491 10155 catalog_manager.cc:4584] TS cee03c4eec03457cb6e7e4d0a42edce9 (127.5.241.65:39477): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d0c723d591dd4aa2b951261b5d77b69a: Network error: Client connection negotiation failed: client connection to 127.5.241.65:39477: connect: Connection refused (error 111)
I20260619 02:39:56.917330 10638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:56.917210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.66:41075 (local address 127.0.0.1:35892)
0619 02:39:56.917257 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:56.917294 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.66:41075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260619 02:39:56.917815 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 10498
I20260619 02:39:56.922436 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 10138
2026-06-19T02:39:56Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2460 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-06-19T02:39:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:39:56Z Disabled control of system clock
I20260619 02:39:56.950079 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:34105
--webserver_interface=127.5.241.126
--webserver_port=0
--builtin_ntp_servers=127.5.241.84:33413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:34105
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 02:39:57.031385 10673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:39:57.031621 10673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:39:57.031643 10673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:39:57.032749 10673 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:39:57.032790 10673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:39:57.032804 10673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:39:57.032816 10673 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:39:57.032828 10673 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:39:57.034217 10673 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:33413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:34105
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:34105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.241.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10673
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:39:57.034426 10673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:39:57.034674 10673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:39:57.039806 10678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:57.039814 10681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:39:57.039944 10673 server_base.cc:1034] running on GCE node
W20260619 02:39:57.039814 10679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:39:57.040310 10673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:39:57.040596 10673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:39:57.041772 10673 hybrid_clock.cc:648] HybridClock initialized: now 1781836797041750 us; error 36 us; skew 500 ppm
I20260619 02:39:57.043183 10673 webserver.cc:460] Webserver started at http://127.5.241.126:45123/ using document root <none> and password file <none>
I20260619 02:39:57.043450 10673 fs_manager.cc:359] Metadata directory not provided
I20260619 02:39:57.043526 10673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:39:57.043678 10673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:39:57.044823 10673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/master-0/data/instance:
uuid: "02e5d5b7ea444f2e954be03ddf07882b"
format_stamp: "Formatted at 2026-06-19 02:39:57 on dist-test-slave-zpbg"
server_key: "bd4680803506f4dc23c24cd1a2a2c830"
server_key_iv: "3e093c56f61b5ef3d80c01000e046ec4"
server_key_version: "encryptionkey@0"
I20260619 02:39:57.045187 10673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/master-0/wal/instance:
uuid: "02e5d5b7ea444f2e954be03ddf07882b"
format_stamp: "Formatted at 2026-06-19 02:39:57 on dist-test-slave-zpbg"
server_key: "bd4680803506f4dc23c24cd1a2a2c830"
server_key_iv: "3e093c56f61b5ef3d80c01000e046ec4"
server_key_version: "encryptionkey@0"
I20260619 02:39:57.046547 10673 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:39:57.047426 10687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:57.047590 10673 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:39:57.047698 10673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "02e5d5b7ea444f2e954be03ddf07882b"
format_stamp: "Formatted at 2026-06-19 02:39:57 on dist-test-slave-zpbg"
server_key: "bd4680803506f4dc23c24cd1a2a2c830"
server_key_iv: "3e093c56f61b5ef3d80c01000e046ec4"
server_key_version: "encryptionkey@0"
I20260619 02:39:57.047775 10673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:39:57.051213 10673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:39:57.051494 10673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:39:57.051637 10673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:39:57.055696 10673 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.126:34105
I20260619 02:39:57.055724 10739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.126:34105 every 8 connection(s)
I20260619 02:39:57.056082 10673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:39:57.056722 10740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:39:57.059376 10740 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02e5d5b7ea444f2e954be03ddf07882b: Bootstrap starting.
I20260619 02:39:57.059906 10740 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 02e5d5b7ea444f2e954be03ddf07882b: Neither blocks nor log segments found. Creating new log.
I20260619 02:39:57.060151 10740 log.cc:824] T 00000000000000000000000000000000 P 02e5d5b7ea444f2e954be03ddf07882b: Log is configured to *not* fsync() on all Append() calls
I20260619 02:39:57.060786 10740 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02e5d5b7ea444f2e954be03ddf07882b: No bootstrap required, opened a new log
I20260619 02:39:57.062292 10740 raft_consensus.cc:348] T 00000000000000000000000000000000 P 02e5d5b7ea444f2e954be03ddf07882b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02e5d5b7ea444f2e954be03ddf07882b" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 34105 } }
I20260619 02:39:57.062427 10740 raft_consensus.cc:374] T 00000000000000000000000000000000 P 02e5d5b7ea444f2e954be03ddf07882b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:39:57.062450 10740 raft_consensus.cc:729] T 00000000000000000000000000000000 P 02e5d5b7ea444f2e954be03ddf07882b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02e5d5b7ea444f2e954be03ddf07882b, State: Initialized, Role: FOLLOWER
I20260619 02:39:57.062552 10740 consensus_queue.cc:260] T 00000000000000000000000000000000 P 02e5d5b7ea444f2e954be03ddf07882b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02e5d5b7ea444f2e954be03ddf07882b" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 34105 } }
I20260619 02:39:57.062641 10740 raft_consensus.cc:388] T 00000000000000000000000000000000 P 02e5d5b7ea444f2e954be03ddf07882b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:39:57.062677 10740 raft_consensus.cc:482] T 00000000000000000000000000000000 P 02e5d5b7ea444f2e954be03ddf07882b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:39:57.062726 10740 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 02e5d5b7ea444f2e954be03ddf07882b [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:39:57.063289 10740 raft_consensus.cc:504] T 00000000000000000000000000000000 P 02e5d5b7ea444f2e954be03ddf07882b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02e5d5b7ea444f2e954be03ddf07882b" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 34105 } }
I20260619 02:39:57.063416 10740 leader_election.cc:302] T 00000000000000000000000000000000 P 02e5d5b7ea444f2e954be03ddf07882b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 02e5d5b7ea444f2e954be03ddf07882b; no voters:
I20260619 02:39:57.063604 10740 leader_election.cc:288] T 00000000000000000000000000000000 P 02e5d5b7ea444f2e954be03ddf07882b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:39:57.063654 10743 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 02e5d5b7ea444f2e954be03ddf07882b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:39:57.063825 10743 raft_consensus.cc:686] T 00000000000000000000000000000000 P 02e5d5b7ea444f2e954be03ddf07882b [term 1 LEADER]: Becoming Leader. State: Replica: 02e5d5b7ea444f2e954be03ddf07882b, State: Running, Role: LEADER
I20260619 02:39:57.063871 10740 sys_catalog.cc:564] T 00000000000000000000000000000000 P 02e5d5b7ea444f2e954be03ddf07882b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:39:57.063923 10743 consensus_queue.cc:237] T 00000000000000000000000000000000 P 02e5d5b7ea444f2e954be03ddf07882b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02e5d5b7ea444f2e954be03ddf07882b" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 34105 } }
I20260619 02:39:57.064301 10744 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02e5d5b7ea444f2e954be03ddf07882b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "02e5d5b7ea444f2e954be03ddf07882b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02e5d5b7ea444f2e954be03ddf07882b" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 34105 } } }
I20260619 02:39:57.064392 10744 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02e5d5b7ea444f2e954be03ddf07882b [sys.catalog]: This master's current role is: LEADER
I20260619 02:39:57.064539 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 10673
I20260619 02:39:57.064649 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/master-0/wal/instance
I20260619 02:39:57.064646 10751 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:39:57.064857 6085 external_mini_cluster.cc:1428] Setting key 976caaaa1f2cdef609e866fb8888e21a
I20260619 02:39:57.065250 10745 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02e5d5b7ea444f2e954be03ddf07882b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 02e5d5b7ea444f2e954be03ddf07882b. Latest consensus state: current_term: 1 leader_uuid: "02e5d5b7ea444f2e954be03ddf07882b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02e5d5b7ea444f2e954be03ddf07882b" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 34105 } } }
I20260619 02:39:57.065323 10745 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02e5d5b7ea444f2e954be03ddf07882b [sys.catalog]: This master's current role is: LEADER
I20260619 02:39:57.065434 10751 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:39:57.067507 10751 catalog_manager.cc:1347] Generated new cluster ID: 5e329978cfa04f4287388730f1fef354
I20260619 02:39:57.067561 10751 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:39:57.067589 10762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:57.065466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47580 (local address 127.5.241.126:34105)
0619 02:39:57.065635 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:57.065645 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:57.065846 (+ 201us) server_negotiation.cc:407] Connection header received
0619 02:39:57.066108 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:57.066116 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:57.066169 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:57.066330 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:57.067105 (+ 775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:57.067108 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:57.067109 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:39:57.067113 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:57.067163 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:39:57.067174 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:57.067251 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:57.067324 (+ 73us) server_negotiation.cc:299] Negotiation successful
0619 02:39:57.067384 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":61,"threads_started":1}
I20260619 02:39:57.076107 10751 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:39:57.076275 10751 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:39:57.081923 10751 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 02e5d5b7ea444f2e954be03ddf07882b: Generated new TSK 0
I20260619 02:39:57.082141 10751 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:39:57.084687 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.65:0
--local_ip_for_outbound_sockets=127.5.241.65
--webserver_interface=127.5.241.65
--webserver_port=0
--tserver_master_addrs=127.5.241.126:34105
--builtin_ntp_servers=127.5.241.84:33413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 02:39:57.184988 10764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:39:57.185237 10764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:39:57.185276 10764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:39:57.186432 10764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:39:57.186496 10764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:39:57.186542 10764 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.65
I20260619 02:39:57.187951 10764 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:33413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.241.65
--webserver_port=0
--tserver_master_addrs=127.5.241.126:34105
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10764
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.65
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:39:57.188233 10764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:39:57.188544 10764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:39:57.194926 10764 server_base.cc:1034] running on GCE node
W20260619 02:39:57.194907 10769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:57.194908 10770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:57.195148 10772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:39:57.195668 10764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:39:57.195997 10764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:39:57.197182 10764 hybrid_clock.cc:648] HybridClock initialized: now 1781836797197178 us; error 42 us; skew 500 ppm
I20260619 02:39:57.198859 10764 webserver.cc:460] Webserver started at http://127.5.241.65:45499/ using document root <none> and password file <none>
I20260619 02:39:57.199159 10764 fs_manager.cc:359] Metadata directory not provided
I20260619 02:39:57.199235 10764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:39:57.199373 10764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:39:57.200980 10764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/ts-0/data/instance:
uuid: "5e3e5cafdada44379cfa8dfd668ccbca"
format_stamp: "Formatted at 2026-06-19 02:39:57 on dist-test-slave-zpbg"
server_key: "39a306c952d72bda3c9414ebfc17c257"
server_key_iv: "2908c38d7935cd0b3bbb448bc4ec53de"
server_key_version: "encryptionkey@0"
I20260619 02:39:57.201493 10764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/ts-0/wal/instance:
uuid: "5e3e5cafdada44379cfa8dfd668ccbca"
format_stamp: "Formatted at 2026-06-19 02:39:57 on dist-test-slave-zpbg"
server_key: "39a306c952d72bda3c9414ebfc17c257"
server_key_iv: "2908c38d7935cd0b3bbb448bc4ec53de"
server_key_version: "encryptionkey@0"
I20260619 02:39:57.203416 10764 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260619 02:39:57.204492 10778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:57.204707 10764 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:39:57.204814 10764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/ts-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/ts-0/wal
uuid: "5e3e5cafdada44379cfa8dfd668ccbca"
format_stamp: "Formatted at 2026-06-19 02:39:57 on dist-test-slave-zpbg"
server_key: "39a306c952d72bda3c9414ebfc17c257"
server_key_iv: "2908c38d7935cd0b3bbb448bc4ec53de"
server_key_version: "encryptionkey@0"
I20260619 02:39:57.204896 10764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:39:57.208300 10764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:39:57.208600 10764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:39:57.208770 10764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:39:57.208983 10764 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:39:57.209300 10764 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:39:57.209352 10764 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:57.209391 10764 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:39:57.209416 10764 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:57.215065 10764 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.65:33641
I20260619 02:39:57.215186 10891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.65:33641 every 8 connection(s)
I20260619 02:39:57.215505 10764 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
I20260619 02:39:57.219137 10762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:57.216409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:48205 (local address 127.5.241.126:34105)
0619 02:39:57.216482 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:57.216485 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:57.216761 (+ 276us) server_negotiation.cc:407] Connection header received
0619 02:39:57.217431 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:57.217432 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:57.217439 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:57.217496 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:57.218878 (+ 1382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:57.218879 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:57.218879 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:39:57.218883 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:57.218899 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:39:57.218903 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:57.218969 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:57.219069 (+ 100us) server_negotiation.cc:299] Negotiation successful
0619 02:39:57.219084 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 02:39:57.219362 10894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:57.216391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:34105 (local address 127.5.241.65:48205)
0619 02:39:57.216620 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:57.216665 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:57.217299 (+ 634us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:57.217568 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:57.217572 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:57.217582 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:39:57.218634 (+ 1052us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:39:57.218635 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:39:57.218673 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:39:57.218710 (+ 37us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:39:57.218980 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:39:57.218986 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:39:57.218989 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:39:57.219075 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 02:39:57.219139 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":103,"threads_started":1}
I20260619 02:39:57.220062 10892 heartbeater.cc:347] Connected to a master server at 127.5.241.126:34105
I20260619 02:39:57.220202 10892 heartbeater.cc:464] Registering TS with master...
I20260619 02:39:57.220374 10892 heartbeater.cc:511] Master 127.5.241.126:34105 requested a full tablet report, sending...
I20260619 02:39:57.220989 10704 ts_manager.cc:194] Registered new tserver with Master: 5e3e5cafdada44379cfa8dfd668ccbca (127.5.241.65:33641)
I20260619 02:39:57.224347 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 10764
I20260619 02:39:57.224588 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/ts-0/wal/instance
I20260619 02:39:57.225008 6085 external_mini_cluster.cc:1428] Setting key 13892ce378fd01f016be3ec1d63de87d
I20260619 02:39:57.226234 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.66:0
--local_ip_for_outbound_sockets=127.5.241.66
--webserver_interface=127.5.241.66
--webserver_port=0
--tserver_master_addrs=127.5.241.126:34105
--builtin_ntp_servers=127.5.241.84:33413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 02:39:57.311307 10895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:39:57.311548 10895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:39:57.311589 10895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:39:57.312814 10895 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:39:57.312882 10895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:39:57.312929 10895 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.66
I20260619 02:39:57.314343 10895 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:33413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.241.66
--webserver_port=0
--tserver_master_addrs=127.5.241.126:34105
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10895
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.66
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:39:57.314625 10895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:39:57.314980 10895 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:39:57.320278 10903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:57.320281 10900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:57.320325 10901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:39:57.320545 10895 server_base.cc:1034] running on GCE node
I20260619 02:39:57.320890 10895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:39:57.321164 10895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:39:57.322355 10895 hybrid_clock.cc:648] HybridClock initialized: now 1781836797322336 us; error 53 us; skew 500 ppm
I20260619 02:39:57.323709 10895 webserver.cc:460] Webserver started at http://127.5.241.66:43047/ using document root <none> and password file <none>
I20260619 02:39:57.323961 10895 fs_manager.cc:359] Metadata directory not provided
I20260619 02:39:57.324069 10895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:39:57.324213 10895 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:39:57.325320 10895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/ts-1/data/instance:
uuid: "f961891d8142431da0846509d2f2f445"
format_stamp: "Formatted at 2026-06-19 02:39:57 on dist-test-slave-zpbg"
server_key: "f54f2b3ed1207e55fd57580bb76223c6"
server_key_iv: "301a6c66e5478160400fca8d31358788"
server_key_version: "encryptionkey@0"
I20260619 02:39:57.325690 10895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/ts-1/wal/instance:
uuid: "f961891d8142431da0846509d2f2f445"
format_stamp: "Formatted at 2026-06-19 02:39:57 on dist-test-slave-zpbg"
server_key: "f54f2b3ed1207e55fd57580bb76223c6"
server_key_iv: "301a6c66e5478160400fca8d31358788"
server_key_version: "encryptionkey@0"
I20260619 02:39:57.327148 10895 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:39:57.327962 10909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:57.328197 10895 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:39:57.328308 10895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/ts-1/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/ts-1/wal
uuid: "f961891d8142431da0846509d2f2f445"
format_stamp: "Formatted at 2026-06-19 02:39:57 on dist-test-slave-zpbg"
server_key: "f54f2b3ed1207e55fd57580bb76223c6"
server_key_iv: "301a6c66e5478160400fca8d31358788"
server_key_version: "encryptionkey@0"
I20260619 02:39:57.328390 10895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:39:57.332021 10895 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:39:57.332314 10895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:39:57.332458 10895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:39:57.332671 10895 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:39:57.332980 10895 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:39:57.333043 10895 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:57.333084 10895 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:39:57.333112 10895 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:57.339154 10895 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.66:44759
I20260619 02:39:57.339195 11022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.66:44759 every 8 connection(s)
I20260619 02:39:57.339543 10895 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
I20260619 02:39:57.342025 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 10895
I20260619 02:39:57.342146 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/ts-1/wal/instance
I20260619 02:39:57.342406 6085 external_mini_cluster.cc:1428] Setting key df650114fb0a547fd77d72219d4809ec
I20260619 02:39:57.342998 10762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:57.340346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:57193 (local address 127.5.241.126:34105)
0619 02:39:57.340428 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:57.340431 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:57.340703 (+ 272us) server_negotiation.cc:407] Connection header received
0619 02:39:57.341184 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:57.341185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:57.341191 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:57.341244 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:57.342687 (+ 1443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:57.342689 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:57.342689 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:39:57.342693 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:57.342711 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:39:57.342715 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:57.342827 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:57.342916 (+ 89us) server_negotiation.cc:299] Negotiation successful
0619 02:39:57.342931 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 02:39:57.343221 11025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:57.340335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:34105 (local address 127.5.241.66:57193)
0619 02:39:57.340572 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:57.340616 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:57.341092 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:57.341298 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:57.341301 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:57.341316 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:39:57.342505 (+ 1189us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:39:57.342506 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:39:57.342548 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:39:57.342586 (+ 38us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:39:57.342828 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:39:57.342832 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:39:57.342835 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:39:57.342908 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 02:39:57.342959 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":97,"threads_started":1}
I20260619 02:39:57.343523 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.67:0
--local_ip_for_outbound_sockets=127.5.241.67
--webserver_interface=127.5.241.67
--webserver_port=0
--tserver_master_addrs=127.5.241.126:34105
--builtin_ntp_servers=127.5.241.84:33413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260619 02:39:57.343835 11023 heartbeater.cc:347] Connected to a master server at 127.5.241.126:34105
I20260619 02:39:57.343963 11023 heartbeater.cc:464] Registering TS with master...
I20260619 02:39:57.344123 11023 heartbeater.cc:511] Master 127.5.241.126:34105 requested a full tablet report, sending...
I20260619 02:39:57.344518 10704 ts_manager.cc:194] Registered new tserver with Master: f961891d8142431da0846509d2f2f445 (127.5.241.66:44759)
W20260619 02:39:57.425783 11026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:39:57.426004 11026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:39:57.426028 11026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:39:57.427516 11026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:39:57.427570 11026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:39:57.427599 11026 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.67
I20260619 02:39:57.429388 11026 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:33413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.241.67
--webserver_port=0
--tserver_master_addrs=127.5.241.126:34105
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11026
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.67
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:39:57.429711 11026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:39:57.430096 11026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:39:57.436146 11031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:57.436210 11032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:57.436110 11034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:39:57.436419 11026 server_base.cc:1034] running on GCE node
I20260619 02:39:57.436676 11026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:39:57.436950 11026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:39:57.438107 11026 hybrid_clock.cc:648] HybridClock initialized: now 1781836797438093 us; error 29 us; skew 500 ppm
I20260619 02:39:57.439519 11026 webserver.cc:460] Webserver started at http://127.5.241.67:41809/ using document root <none> and password file <none>
I20260619 02:39:57.439800 11026 fs_manager.cc:359] Metadata directory not provided
I20260619 02:39:57.439877 11026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:39:57.440057 11026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:39:57.441205 11026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/ts-2/data/instance:
uuid: "e6574dcec12042e1ba185f02542c2bcc"
format_stamp: "Formatted at 2026-06-19 02:39:57 on dist-test-slave-zpbg"
server_key: "8babce1f25c8b4030f8f354f8f10b9ac"
server_key_iv: "a0301995eb03e108e6f76579b64d31c4"
server_key_version: "encryptionkey@0"
I20260619 02:39:57.441602 11026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/ts-2/wal/instance:
uuid: "e6574dcec12042e1ba185f02542c2bcc"
format_stamp: "Formatted at 2026-06-19 02:39:57 on dist-test-slave-zpbg"
server_key: "8babce1f25c8b4030f8f354f8f10b9ac"
server_key_iv: "a0301995eb03e108e6f76579b64d31c4"
server_key_version: "encryptionkey@0"
I20260619 02:39:57.443115 11026 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:39:57.443898 11040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:57.444182 11026 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:39:57.444303 11026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/ts-2/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/ts-2/wal
uuid: "e6574dcec12042e1ba185f02542c2bcc"
format_stamp: "Formatted at 2026-06-19 02:39:57 on dist-test-slave-zpbg"
server_key: "8babce1f25c8b4030f8f354f8f10b9ac"
server_key_iv: "a0301995eb03e108e6f76579b64d31c4"
server_key_version: "encryptionkey@0"
I20260619 02:39:57.444386 11026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:39:57.447870 11026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:39:57.448154 11026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:39:57.448299 11026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:39:57.448628 11026 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:39:57.449187 11026 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:39:57.449252 11026 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:57.449287 11026 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:39:57.449335 11026 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:57.455278 11026 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.67:43683
I20260619 02:39:57.455319 11153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.67:43683 every 8 connection(s)
I20260619 02:39:57.455783 11026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
I20260619 02:39:57.458958 10762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:57.456604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:41181 (local address 127.5.241.126:34105)
0619 02:39:57.456665 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:57.456667 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:57.456885 (+ 218us) server_negotiation.cc:407] Connection header received
0619 02:39:57.457408 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:57.457409 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:57.457415 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:57.457467 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:57.458670 (+ 1203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:57.458672 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:57.458672 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:39:57.458677 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:57.458693 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:39:57.458698 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:57.458831 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:57.458869 (+ 38us) server_negotiation.cc:299] Negotiation successful
0619 02:39:57.458886 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 02:39:57.459184 11156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:57.456578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:34105 (local address 127.5.241.67:41181)
0619 02:39:57.456767 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:57.456802 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:57.457295 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:57.457559 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:57.457568 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:57.457576 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:39:57.458509 (+ 933us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:39:57.458510 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:39:57.458538 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:39:57.458567 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:39:57.458788 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:39:57.458794 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:39:57.458798 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:39:57.458875 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 02:39:57.458938 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":81,"threads_started":1}
I20260619 02:39:57.459203 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 11026
I20260619 02:39:57.459303 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/ts-2/wal/instance
I20260619 02:39:57.459507 6085 external_mini_cluster.cc:1428] Setting key a181e4350fe29e2925a51f65a53a9386
I20260619 02:39:57.459738 11154 heartbeater.cc:347] Connected to a master server at 127.5.241.126:34105
I20260619 02:39:57.459928 11154 heartbeater.cc:464] Registering TS with master...
I20260619 02:39:57.460096 11154 heartbeater.cc:511] Master 127.5.241.126:34105 requested a full tablet report, sending...
I20260619 02:39:57.460435 10703 ts_manager.cc:194] Registered new tserver with Master: e6574dcec12042e1ba185f02542c2bcc (127.5.241.67:43683)
I20260619 02:39:57.471704 6085 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:39:57.474453 10762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:57.473131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47594 (local address 127.5.241.126:34105)
0619 02:39:57.473203 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:57.473205 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:57.473316 (+ 111us) server_negotiation.cc:407] Connection header received
0619 02:39:57.473398 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:57.473399 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:57.473405 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:57.473479 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:57.474167 (+ 688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:57.474169 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:57.474169 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:39:57.474174 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:57.474200 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:39:57.474204 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:57.474269 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:57.474329 (+ 60us) server_negotiation.cc:299] Negotiation successful
0619 02:39:57.474348 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 02:39:57.476362 10703 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47594:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:39:57.477909 10703 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:39:57.481873 11165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:57.480424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54106 (local address 127.5.241.67:43683)
0619 02:39:57.480636 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:57.480642 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:57.480652 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:39:57.480680 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:57.480682 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:57.480693 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:57.480765 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:57.481517 (+ 752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:57.481521 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:57.481521 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:39:57.481524 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:57.481551 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:39:57.481561 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:57.481655 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:57.481769 (+ 114us) server_negotiation.cc:299] Negotiation successful
0619 02:39:57.481786 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":96,"threads_started":1}
I20260619 02:39:57.481861 11163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:57.480249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.67:43683 (local address 127.0.0.1:54106)
0619 02:39:57.480423 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:57.480464 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:57.480519 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:57.480769 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:57.480774 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:57.480780 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:39:57.481281 (+ 501us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:39:57.481282 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:39:57.481299 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:39:57.481325 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:39:57.481653 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:39:57.481660 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:39:57.481663 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:39:57.481740 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 02:39:57.481755 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":74,"threads_started":1}
I20260619 02:39:57.482007 11166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:57.480438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55982 (local address 127.5.241.66:44759)
0619 02:39:57.480736 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:57.480741 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:57.480752 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:39:57.480777 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:57.480779 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:57.480787 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:57.480858 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:57.481526 (+ 668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:57.481535 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:57.481535 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:39:57.481539 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:57.481572 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:39:57.481608 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:57.481682 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:57.481906 (+ 224us) server_negotiation.cc:299] Negotiation successful
0619 02:39:57.481921 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":86,"threads_started":1}
I20260619 02:39:57.481871 11164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:57.480249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.66:44759 (local address 127.0.0.1:55982)
0619 02:39:57.480644 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:57.480656 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:57.480709 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:57.481030 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:57.481030 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:57.481033 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:39:57.481428 (+ 395us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:39:57.481428 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:39:57.481434 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:39:57.481445 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:39:57.481675 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:39:57.481676 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:39:57.481676 (+ 0us) client_negotiation.cc:769] Sending connection context
0619 02:39:57.481761 (+ 85us) client_negotiation.cc:240] Negotiation successful
0619 02:39:57.481776 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"mutex_wait_us":3,"thread_start_us":60,"threads_started":1}
I20260619 02:39:57.482841 11162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:57.480292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:33641 (local address 127.0.0.1:60660)
0619 02:39:57.481708 (+ 1416us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:57.481719 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:57.481763 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:57.482009 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:57.482010 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:57.482012 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:39:57.482433 (+ 421us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:39:57.482433 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:39:57.482439 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:39:57.482452 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:39:57.482701 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:39:57.482701 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:39:57.482702 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 02:39:57.482744 (+ 42us) client_negotiation.cc:240] Negotiation successful
0619 02:39:57.482757 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1403,"thread_start_us":88,"threads_started":1}
I20260619 02:39:57.483070 11167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:57.480511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60660 (local address 127.5.241.65:33641)
0619 02:39:57.480695 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:57.480698 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:57.481821 (+ 1123us) server_negotiation.cc:407] Connection header received
0619 02:39:57.481852 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:57.481855 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:57.481864 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:57.481929 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:57.482533 (+ 604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:57.482539 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:57.482539 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:39:57.482544 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:57.482582 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:39:57.482604 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:57.482675 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:57.482978 (+ 303us) server_negotiation.cc:299] Negotiation successful
0619 02:39:57.482994 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":105,"threads_started":1}
I20260619 02:39:57.483220 11088 tablet_service.cc:1467] Processing CreateTablet for tablet f6c1e89efee44769b95ddad30b78bfb7 (DEFAULT_TABLE table=test-table [id=271853adb20e47d8bfad81164e9cc032]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:39:57.483219 10957 tablet_service.cc:1467] Processing CreateTablet for tablet f6c1e89efee44769b95ddad30b78bfb7 (DEFAULT_TABLE table=test-table [id=271853adb20e47d8bfad81164e9cc032]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:39:57.483604 11088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6c1e89efee44769b95ddad30b78bfb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:39:57.483605 10957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6c1e89efee44769b95ddad30b78bfb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:39:57.483585 10826 tablet_service.cc:1467] Processing CreateTablet for tablet f6c1e89efee44769b95ddad30b78bfb7 (DEFAULT_TABLE table=test-table [id=271853adb20e47d8bfad81164e9cc032]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:39:57.483891 10826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6c1e89efee44769b95ddad30b78bfb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:39:57.486338 11168 tablet_bootstrap.cc:492] T f6c1e89efee44769b95ddad30b78bfb7 P e6574dcec12042e1ba185f02542c2bcc: Bootstrap starting.
I20260619 02:39:57.486902 11170 tablet_bootstrap.cc:492] T f6c1e89efee44769b95ddad30b78bfb7 P 5e3e5cafdada44379cfa8dfd668ccbca: Bootstrap starting.
I20260619 02:39:57.487176 11168 tablet_bootstrap.cc:654] T f6c1e89efee44769b95ddad30b78bfb7 P e6574dcec12042e1ba185f02542c2bcc: Neither blocks nor log segments found. Creating new log.
I20260619 02:39:57.487509 11168 log.cc:824] T f6c1e89efee44769b95ddad30b78bfb7 P e6574dcec12042e1ba185f02542c2bcc: Log is configured to *not* fsync() on all Append() calls
I20260619 02:39:57.487567 11170 tablet_bootstrap.cc:654] T f6c1e89efee44769b95ddad30b78bfb7 P 5e3e5cafdada44379cfa8dfd668ccbca: Neither blocks nor log segments found. Creating new log.
I20260619 02:39:57.487900 11170 log.cc:824] T f6c1e89efee44769b95ddad30b78bfb7 P 5e3e5cafdada44379cfa8dfd668ccbca: Log is configured to *not* fsync() on all Append() calls
I20260619 02:39:57.488253 11169 tablet_bootstrap.cc:492] T f6c1e89efee44769b95ddad30b78bfb7 P f961891d8142431da0846509d2f2f445: Bootstrap starting.
I20260619 02:39:57.488618 11168 tablet_bootstrap.cc:492] T f6c1e89efee44769b95ddad30b78bfb7 P e6574dcec12042e1ba185f02542c2bcc: No bootstrap required, opened a new log
I20260619 02:39:57.488693 11168 ts_tablet_manager.cc:1397] T f6c1e89efee44769b95ddad30b78bfb7 P e6574dcec12042e1ba185f02542c2bcc: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 02:39:57.488744 11170 tablet_bootstrap.cc:492] T f6c1e89efee44769b95ddad30b78bfb7 P 5e3e5cafdada44379cfa8dfd668ccbca: No bootstrap required, opened a new log
I20260619 02:39:57.488862 11170 ts_tablet_manager.cc:1397] T f6c1e89efee44769b95ddad30b78bfb7 P 5e3e5cafdada44379cfa8dfd668ccbca: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 02:39:57.488897 11169 tablet_bootstrap.cc:654] T f6c1e89efee44769b95ddad30b78bfb7 P f961891d8142431da0846509d2f2f445: Neither blocks nor log segments found. Creating new log.
I20260619 02:39:57.489192 11169 log.cc:824] T f6c1e89efee44769b95ddad30b78bfb7 P f961891d8142431da0846509d2f2f445: Log is configured to *not* fsync() on all Append() calls
I20260619 02:39:57.489995 11169 tablet_bootstrap.cc:492] T f6c1e89efee44769b95ddad30b78bfb7 P f961891d8142431da0846509d2f2f445: No bootstrap required, opened a new log
I20260619 02:39:57.490067 11169 ts_tablet_manager.cc:1397] T f6c1e89efee44769b95ddad30b78bfb7 P f961891d8142431da0846509d2f2f445: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:39:57.491251 11170 raft_consensus.cc:348] T f6c1e89efee44769b95ddad30b78bfb7 P 5e3e5cafdada44379cfa8dfd668ccbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e3e5cafdada44379cfa8dfd668ccbca" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 33641 } } peers { permanent_uuid: "f961891d8142431da0846509d2f2f445" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 44759 } } peers { permanent_uuid: "e6574dcec12042e1ba185f02542c2bcc" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 43683 } }
I20260619 02:39:57.491420 11170 raft_consensus.cc:374] T f6c1e89efee44769b95ddad30b78bfb7 P 5e3e5cafdada44379cfa8dfd668ccbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:39:57.491470 11170 raft_consensus.cc:729] T f6c1e89efee44769b95ddad30b78bfb7 P 5e3e5cafdada44379cfa8dfd668ccbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e3e5cafdada44379cfa8dfd668ccbca, State: Initialized, Role: FOLLOWER
I20260619 02:39:57.491593 11170 consensus_queue.cc:260] T f6c1e89efee44769b95ddad30b78bfb7 P 5e3e5cafdada44379cfa8dfd668ccbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e3e5cafdada44379cfa8dfd668ccbca" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 33641 } } peers { permanent_uuid: "f961891d8142431da0846509d2f2f445" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 44759 } } peers { permanent_uuid: "e6574dcec12042e1ba185f02542c2bcc" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 43683 } }
I20260619 02:39:57.491921 11170 ts_tablet_manager.cc:1428] T f6c1e89efee44769b95ddad30b78bfb7 P 5e3e5cafdada44379cfa8dfd668ccbca: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260619 02:39:57.491420 11168 raft_consensus.cc:348] T f6c1e89efee44769b95ddad30b78bfb7 P e6574dcec12042e1ba185f02542c2bcc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e3e5cafdada44379cfa8dfd668ccbca" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 33641 } } peers { permanent_uuid: "f961891d8142431da0846509d2f2f445" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 44759 } } peers { permanent_uuid: "e6574dcec12042e1ba185f02542c2bcc" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 43683 } }
I20260619 02:39:57.491993 11168 raft_consensus.cc:374] T f6c1e89efee44769b95ddad30b78bfb7 P e6574dcec12042e1ba185f02542c2bcc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:39:57.492043 11168 raft_consensus.cc:729] T f6c1e89efee44769b95ddad30b78bfb7 P e6574dcec12042e1ba185f02542c2bcc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6574dcec12042e1ba185f02542c2bcc, State: Initialized, Role: FOLLOWER
I20260619 02:39:57.492076 10892 heartbeater.cc:503] Master 127.5.241.126:34105 was elected leader, sending a full tablet report...
I20260619 02:39:57.492189 11168 consensus_queue.cc:260] T f6c1e89efee44769b95ddad30b78bfb7 P e6574dcec12042e1ba185f02542c2bcc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e3e5cafdada44379cfa8dfd668ccbca" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 33641 } } peers { permanent_uuid: "f961891d8142431da0846509d2f2f445" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 44759 } } peers { permanent_uuid: "e6574dcec12042e1ba185f02542c2bcc" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 43683 } }
I20260619 02:39:57.492467 11168 ts_tablet_manager.cc:1428] T f6c1e89efee44769b95ddad30b78bfb7 P e6574dcec12042e1ba185f02542c2bcc: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260619 02:39:57.492588 11154 heartbeater.cc:503] Master 127.5.241.126:34105 was elected leader, sending a full tablet report...
I20260619 02:39:57.492769 11169 raft_consensus.cc:348] T f6c1e89efee44769b95ddad30b78bfb7 P f961891d8142431da0846509d2f2f445 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e3e5cafdada44379cfa8dfd668ccbca" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 33641 } } peers { permanent_uuid: "f961891d8142431da0846509d2f2f445" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 44759 } } peers { permanent_uuid: "e6574dcec12042e1ba185f02542c2bcc" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 43683 } }
I20260619 02:39:57.492918 11169 raft_consensus.cc:374] T f6c1e89efee44769b95ddad30b78bfb7 P f961891d8142431da0846509d2f2f445 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:39:57.492960 11169 raft_consensus.cc:729] T f6c1e89efee44769b95ddad30b78bfb7 P f961891d8142431da0846509d2f2f445 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f961891d8142431da0846509d2f2f445, State: Initialized, Role: FOLLOWER
I20260619 02:39:57.493073 11169 consensus_queue.cc:260] T f6c1e89efee44769b95ddad30b78bfb7 P f961891d8142431da0846509d2f2f445 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e3e5cafdada44379cfa8dfd668ccbca" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 33641 } } peers { permanent_uuid: "f961891d8142431da0846509d2f2f445" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 44759 } } peers { permanent_uuid: "e6574dcec12042e1ba185f02542c2bcc" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 43683 } }
I20260619 02:39:57.493322 11169 ts_tablet_manager.cc:1428] T f6c1e89efee44769b95ddad30b78bfb7 P f961891d8142431da0846509d2f2f445: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260619 02:39:57.493397 11023 heartbeater.cc:503] Master 127.5.241.126:34105 was elected leader, sending a full tablet report...
W20260619 02:39:57.590171 11024 tablet.cc:2369] T f6c1e89efee44769b95ddad30b78bfb7 P f961891d8142431da0846509d2f2f445: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:39:57.706401 11155 tablet.cc:2369] T f6c1e89efee44769b95ddad30b78bfb7 P e6574dcec12042e1ba185f02542c2bcc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:39:57.716298 10893 tablet.cc:2369] T f6c1e89efee44769b95ddad30b78bfb7 P 5e3e5cafdada44379cfa8dfd668ccbca: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:39:57.739542 11174 raft_consensus.cc:482] T f6c1e89efee44769b95ddad30b78bfb7 P 5e3e5cafdada44379cfa8dfd668ccbca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:39:57.739723 11174 raft_consensus.cc:504] T f6c1e89efee44769b95ddad30b78bfb7 P 5e3e5cafdada44379cfa8dfd668ccbca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e3e5cafdada44379cfa8dfd668ccbca" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 33641 } } peers { permanent_uuid: "f961891d8142431da0846509d2f2f445" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 44759 } } peers { permanent_uuid: "e6574dcec12042e1ba185f02542c2bcc" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 43683 } }
I20260619 02:39:57.740094 11174 leader_election.cc:288] T f6c1e89efee44769b95ddad30b78bfb7 P 5e3e5cafdada44379cfa8dfd668ccbca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f961891d8142431da0846509d2f2f445 (127.5.241.66:44759), e6574dcec12042e1ba185f02542c2bcc (127.5.241.67:43683)
I20260619 02:39:57.741621 11177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:57.740339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.67:43683 (local address 127.5.241.65:38345)
0619 02:39:57.740485 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:57.740500 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:57.740602 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:57.740752 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:57.740753 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:57.740756 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:39:57.741242 (+ 486us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:39:57.741242 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:39:57.741249 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:39:57.741260 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:39:57.741436 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:39:57.741437 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:39:57.741437 (+ 0us) client_negotiation.cc:769] Sending connection context
0619 02:39:57.741491 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 02:39:57.741506 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":63,"threads_started":1}
I20260619 02:39:57.741621 11165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:57.740426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:38345 (local address 127.5.241.67:43683)
0619 02:39:57.740476 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:57.740478 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:57.740601 (+ 123us) server_negotiation.cc:407] Connection header received
0619 02:39:57.740665 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:57.740666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:57.740673 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:57.740736 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:57.741342 (+ 606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:57.741344 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:57.741344 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:39:57.741348 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:57.741370 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:39:57.741375 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:57.741441 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:57.741516 (+ 75us) server_negotiation.cc:299] Negotiation successful
0619 02:39:57.741531 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 02:39:57.741753 11166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:57.740496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:38449 (local address 127.5.241.66:44759)
0619 02:39:57.740582 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:57.740584 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:57.740643 (+ 59us) server_negotiation.cc:407] Connection header received
0619 02:39:57.740742 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:57.740743 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:57.740748 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:57.740799 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:57.741501 (+ 702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:57.741503 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:57.741503 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:39:57.741507 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:57.741519 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:39:57.741524 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:57.741591 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:57.741659 (+ 68us) server_negotiation.cc:299] Negotiation successful
0619 02:39:57.741674 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 02:39:57.741915 11178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:57.740342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.66:44759 (local address 127.5.241.65:38449)
0619 02:39:57.740540 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:57.740551 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:57.740655 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:57.740819 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:57.740819 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:57.740821 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:39:57.741402 (+ 581us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:39:57.741402 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:39:57.741409 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:39:57.741422 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:39:57.741601 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:39:57.741602 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:39:57.741602 (+ 0us) client_negotiation.cc:769] Sending connection context
0619 02:39:57.741651 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 02:39:57.741664 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":66,"threads_started":1}
I20260619 02:39:57.741943 11108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6c1e89efee44769b95ddad30b78bfb7" candidate_uuid: "5e3e5cafdada44379cfa8dfd668ccbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6574dcec12042e1ba185f02542c2bcc" is_pre_election: true
I20260619 02:39:57.742080 11108 raft_consensus.cc:2454] T f6c1e89efee44769b95ddad30b78bfb7 P e6574dcec12042e1ba185f02542c2bcc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e3e5cafdada44379cfa8dfd668ccbca in term 0.
I20260619 02:39:57.742313 10780 leader_election.cc:302] T f6c1e89efee44769b95ddad30b78bfb7 P 5e3e5cafdada44379cfa8dfd668ccbca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e3e5cafdada44379cfa8dfd668ccbca, e6574dcec12042e1ba185f02542c2bcc; no voters:
I20260619 02:39:57.742319 10977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6c1e89efee44769b95ddad30b78bfb7" candidate_uuid: "5e3e5cafdada44379cfa8dfd668ccbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f961891d8142431da0846509d2f2f445" is_pre_election: true
I20260619 02:39:57.742449 10977 raft_consensus.cc:2454] T f6c1e89efee44769b95ddad30b78bfb7 P f961891d8142431da0846509d2f2f445 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e3e5cafdada44379cfa8dfd668ccbca in term 0.
I20260619 02:39:57.742527 11174 raft_consensus.cc:2781] T f6c1e89efee44769b95ddad30b78bfb7 P 5e3e5cafdada44379cfa8dfd668ccbca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:39:57.742616 11174 raft_consensus.cc:482] T f6c1e89efee44769b95ddad30b78bfb7 P 5e3e5cafdada44379cfa8dfd668ccbca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:39:57.742651 11174 raft_consensus.cc:3037] T f6c1e89efee44769b95ddad30b78bfb7 P 5e3e5cafdada44379cfa8dfd668ccbca [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:39:57.743445 11174 raft_consensus.cc:504] T f6c1e89efee44769b95ddad30b78bfb7 P 5e3e5cafdada44379cfa8dfd668ccbca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e3e5cafdada44379cfa8dfd668ccbca" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 33641 } } peers { permanent_uuid: "f961891d8142431da0846509d2f2f445" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 44759 } } peers { permanent_uuid: "e6574dcec12042e1ba185f02542c2bcc" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 43683 } }
I20260619 02:39:57.743592 11174 leader_election.cc:288] T f6c1e89efee44769b95ddad30b78bfb7 P 5e3e5cafdada44379cfa8dfd668ccbca [CANDIDATE]: Term 1 election: Requested vote from peers f961891d8142431da0846509d2f2f445 (127.5.241.66:44759), e6574dcec12042e1ba185f02542c2bcc (127.5.241.67:43683)
I20260619 02:39:57.743731 10977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6c1e89efee44769b95ddad30b78bfb7" candidate_uuid: "5e3e5cafdada44379cfa8dfd668ccbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f961891d8142431da0846509d2f2f445"
I20260619 02:39:57.743736 11108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6c1e89efee44769b95ddad30b78bfb7" candidate_uuid: "5e3e5cafdada44379cfa8dfd668ccbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6574dcec12042e1ba185f02542c2bcc"
I20260619 02:39:57.743809 11108 raft_consensus.cc:3037] T f6c1e89efee44769b95ddad30b78bfb7 P e6574dcec12042e1ba185f02542c2bcc [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:39:57.743834 10977 raft_consensus.cc:3037] T f6c1e89efee44769b95ddad30b78bfb7 P f961891d8142431da0846509d2f2f445 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:39:57.744509 10977 raft_consensus.cc:2454] T f6c1e89efee44769b95ddad30b78bfb7 P f961891d8142431da0846509d2f2f445 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e3e5cafdada44379cfa8dfd668ccbca in term 1.
I20260619 02:39:57.744554 11108 raft_consensus.cc:2454] T f6c1e89efee44769b95ddad30b78bfb7 P e6574dcec12042e1ba185f02542c2bcc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e3e5cafdada44379cfa8dfd668ccbca in term 1.
I20260619 02:39:57.744711 10779 leader_election.cc:302] T f6c1e89efee44769b95ddad30b78bfb7 P 5e3e5cafdada44379cfa8dfd668ccbca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e3e5cafdada44379cfa8dfd668ccbca, f961891d8142431da0846509d2f2f445; no voters:
I20260619 02:39:57.744815 11174 raft_consensus.cc:2781] T f6c1e89efee44769b95ddad30b78bfb7 P 5e3e5cafdada44379cfa8dfd668ccbca [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:39:57.744962 11174 raft_consensus.cc:686] T f6c1e89efee44769b95ddad30b78bfb7 P 5e3e5cafdada44379cfa8dfd668ccbca [term 1 LEADER]: Becoming Leader. State: Replica: 5e3e5cafdada44379cfa8dfd668ccbca, State: Running, Role: LEADER
I20260619 02:39:57.745096 11174 consensus_queue.cc:237] T f6c1e89efee44769b95ddad30b78bfb7 P 5e3e5cafdada44379cfa8dfd668ccbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e3e5cafdada44379cfa8dfd668ccbca" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 33641 } } peers { permanent_uuid: "f961891d8142431da0846509d2f2f445" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 44759 } } peers { permanent_uuid: "e6574dcec12042e1ba185f02542c2bcc" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 43683 } }
I20260619 02:39:57.745967 10704 catalog_manager.cc:5495] T f6c1e89efee44769b95ddad30b78bfb7 P 5e3e5cafdada44379cfa8dfd668ccbca reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e3e5cafdada44379cfa8dfd668ccbca (127.5.241.65). New cstate: current_term: 1 leader_uuid: "5e3e5cafdada44379cfa8dfd668ccbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e3e5cafdada44379cfa8dfd668ccbca" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 33641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f961891d8142431da0846509d2f2f445" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 44759 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6574dcec12042e1ba185f02542c2bcc" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 43683 } health_report { overall_health: UNKNOWN } } }
I20260619 02:39:57.766278 11167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:57.765247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60662 (local address 127.5.241.65:33641)
0619 02:39:57.765295 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:57.765297 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:57.765305 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:39:57.765348 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:57.765348 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:57.765360 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:57.765426 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:57.766012 (+ 586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:57.766013 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:57.766013 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:39:57.766018 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:57.766039 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:39:57.766043 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:57.766123 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:57.766181 (+ 58us) server_negotiation.cc:299] Negotiation successful
0619 02:39:57.766195 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 02:39:57.768112 11108 raft_consensus.cc:1261] T f6c1e89efee44769b95ddad30b78bfb7 P e6574dcec12042e1ba185f02542c2bcc [term 1 FOLLOWER]: Refusing update from remote peer 5e3e5cafdada44379cfa8dfd668ccbca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:39:57.768163 10977 raft_consensus.cc:1261] T f6c1e89efee44769b95ddad30b78bfb7 P f961891d8142431da0846509d2f2f445 [term 1 FOLLOWER]: Refusing update from remote peer 5e3e5cafdada44379cfa8dfd668ccbca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:39:57.768469 11174 consensus_queue.cc:1035] T f6c1e89efee44769b95ddad30b78bfb7 P 5e3e5cafdada44379cfa8dfd668ccbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6574dcec12042e1ba185f02542c2bcc" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 43683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:39:57.768666 11174 consensus_queue.cc:1035] T f6c1e89efee44769b95ddad30b78bfb7 P 5e3e5cafdada44379cfa8dfd668ccbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "f961891d8142431da0846509d2f2f445" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 44759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:39:57.771376 11184 mvcc.cc:205] Tried to move back new op lower bound from 7298403523655593984 to 7298403523564519424. Current Snapshot: MvccSnapshot[applied={T|T < 7298403523655593984}]
I20260619 02:39:57.771821 11183 mvcc.cc:205] Tried to move back new op lower bound from 7298403523655593984 to 7298403523564519424. Current Snapshot: MvccSnapshot[applied={T|T < 7298403523655593984 or (T in {7298403523655593984})}]
I20260619 02:39:57.773342 10704 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47594:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:39:57.773437 10704 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47594:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:39:57.774231 10704 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 02e5d5b7ea444f2e954be03ddf07882b: Sending DeleteTablet for 3 replicas of tablet f6c1e89efee44769b95ddad30b78bfb7
I20260619 02:39:57.774564 11088 tablet_service.cc:1514] Processing DeleteTablet for tablet f6c1e89efee44769b95ddad30b78bfb7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:39:57 UTC) from {username='slave'} at 127.0.0.1:54106
I20260619 02:39:57.774609 10826 tablet_service.cc:1514] Processing DeleteTablet for tablet f6c1e89efee44769b95ddad30b78bfb7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:39:57 UTC) from {username='slave'} at 127.0.0.1:60660
I20260619 02:39:57.774670 10957 tablet_service.cc:1514] Processing DeleteTablet for tablet f6c1e89efee44769b95ddad30b78bfb7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:39:57 UTC) from {username='slave'} at 127.0.0.1:55982
I20260619 02:39:57.774820 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 10764
I20260619 02:39:57.774888 11192 tablet_replica.cc:330] T f6c1e89efee44769b95ddad30b78bfb7 P e6574dcec12042e1ba185f02542c2bcc: stopping tablet replica
I20260619 02:39:57.775440 11192 raft_consensus.cc:2229] T f6c1e89efee44769b95ddad30b78bfb7 P e6574dcec12042e1ba185f02542c2bcc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:39:57.775539 11192 raft_consensus.cc:2258] T f6c1e89efee44769b95ddad30b78bfb7 P e6574dcec12042e1ba185f02542c2bcc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:39:57.775949 11192 ts_tablet_manager.cc:1905] T f6c1e89efee44769b95ddad30b78bfb7 P e6574dcec12042e1ba185f02542c2bcc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:39:57.777670 11192 ts_tablet_manager.cc:1918] T f6c1e89efee44769b95ddad30b78bfb7 P e6574dcec12042e1ba185f02542c2bcc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:39:57.777741 11192 log.cc:1196] T f6c1e89efee44769b95ddad30b78bfb7 P e6574dcec12042e1ba185f02542c2bcc: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836772852275-6085-0/minicluster-data/ts-2/wal/wals/f6c1e89efee44769b95ddad30b78bfb7
I20260619 02:39:57.777974 11192 ts_tablet_manager.cc:1939] T f6c1e89efee44769b95ddad30b78bfb7 P e6574dcec12042e1ba185f02542c2bcc: Deleting consensus metadata
I20260619 02:39:57.778370 10689 catalog_manager.cc:4841] TS e6574dcec12042e1ba185f02542c2bcc (127.5.241.67:43683): tablet f6c1e89efee44769b95ddad30b78bfb7 (table test-table [id=271853adb20e47d8bfad81164e9cc032]) successfully deleted
I20260619 02:39:57.778887 11194 tablet_replica.cc:330] T f6c1e89efee44769b95ddad30b78bfb7 P f961891d8142431da0846509d2f2f445: stopping tablet replica
I20260619 02:39:57.778993 11194 raft_consensus.cc:2229] T f6c1e89efee44769b95ddad30b78bfb7 P f961891d8142431da0846509d2f2f445 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:39:57.779098 11194 raft_consensus.cc:2258] T f6c1e89efee44769b95ddad30b78bfb7 P f961891d8142431da0846509d2f2f445 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:39:57.779487 11194 ts_tablet_manager.cc:1905] T f6c1e89efee44769b95ddad30b78bfb7 P f961891d8142431da0846509d2f2f445: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 02:39:57.780797 10690 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.241.65:33641 (error 108)
I20260619 02:39:57.780817 11194 ts_tablet_manager.cc:1918] T f6c1e89efee44769b95ddad30b78bfb7 P f961891d8142431da0846509d2f2f445: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:39:57.780967 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 10895
I20260619 02:39:57.781592 11162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:57.781108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:33641 (local address 127.0.0.1:60668)
0619 02:39:57.781505 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:57.781553 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.65:33641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374}
W20260619 02:39:57.781738 10690 catalog_manager.cc:4584] TS 5e3e5cafdada44379cfa8dfd668ccbca (127.5.241.65:33641): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f6c1e89efee44769b95ddad30b78bfb7: Network error: Client connection negotiation failed: client connection to 127.5.241.65:33641: connect: Connection refused (error 111)
I20260619 02:39:57.786204 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 11026
I20260619 02:39:57.786279 11162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:57.786156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.66:44759 (local address 127.0.0.1:55990)
0619 02:39:57.786220 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:57.786249 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.66:44759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260619 02:39:57.790355 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 10673
2026-06-19T02:39:57Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (866 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260619 02:39:57.809770 6085 mini_oidc.cc:152] Starting JWKS server
I20260619 02:39:57.811062 6085 webserver.cc:460] Webserver started at https://127.0.0.1:37101/ using document root <none> and password file <none>
I20260619 02:39:57.811208 6085 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 02:39:57.811393 6085 webserver.cc:460] Webserver started at http://127.0.0.1:42089/ using document root <none> and password file <none>
2026-06-19T02:39:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:39:57Z Disabled control of system clock
I20260619 02:39:57.819068 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:44439
--webserver_interface=127.5.241.126
--webserver_port=0
--builtin_ntp_servers=127.5.241.84:35821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:44439
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37101/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836772852275-6085-0/testchainca.cert with env {}
W20260619 02:39:57.899590 11201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:39:57.899812 11201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:39:57.899834 11201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:39:57.901036 11201 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:39:57.901075 11201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:39:57.901088 11201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:39:57.901101 11201 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:39:57.901114 11201 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 02:39:57.901126 11201 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 02:39:57.901180 11201 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37101/jwks.json
I20260619 02:39:57.902731 11201 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:35821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836772852275-6085-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:44439
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:44439
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37101/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.241.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836772852275-6085-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:39:57.903019 11201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:39:57.903192 11201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:39:57.908195 11209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:39:57.908269 11201 server_base.cc:1034] running on GCE node
W20260619 02:39:57.908219 11207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:57.908212 11206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:39:57.908707 11201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:39:57.908995 11201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:39:57.910195 11201 hybrid_clock.cc:648] HybridClock initialized: now 1781836797910174 us; error 38 us; skew 500 ppm
I20260619 02:39:57.911592 11201 webserver.cc:460] Webserver started at http://127.5.241.126:33881/ using document root <none> and password file <none>
I20260619 02:39:57.911844 11201 fs_manager.cc:359] Metadata directory not provided
I20260619 02:39:57.911903 11201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:39:57.912025 11201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:39:57.913120 11201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836772852275-6085-0/minicluster-data/master-0/data/instance:
uuid: "8d6aba8fe5394b6fb3053676ad7438c9"
format_stamp: "Formatted at 2026-06-19 02:39:57 on dist-test-slave-zpbg"
server_key: "8d7a3435283760c4bdfabf7b8ca7eb17"
server_key_iv: "4e55bed339aa1c28aac4913bcd80ac3b"
server_key_version: "encryptionkey@0"
I20260619 02:39:57.913487 11201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836772852275-6085-0/minicluster-data/master-0/wal/instance:
uuid: "8d6aba8fe5394b6fb3053676ad7438c9"
format_stamp: "Formatted at 2026-06-19 02:39:57 on dist-test-slave-zpbg"
server_key: "8d7a3435283760c4bdfabf7b8ca7eb17"
server_key_iv: "4e55bed339aa1c28aac4913bcd80ac3b"
server_key_version: "encryptionkey@0"
I20260619 02:39:57.914918 11201 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:39:57.915719 11215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:57.915907 11201 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:39:57.916002 11201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "8d6aba8fe5394b6fb3053676ad7438c9"
format_stamp: "Formatted at 2026-06-19 02:39:57 on dist-test-slave-zpbg"
server_key: "8d7a3435283760c4bdfabf7b8ca7eb17"
server_key_iv: "4e55bed339aa1c28aac4913bcd80ac3b"
server_key_version: "encryptionkey@0"
I20260619 02:39:57.916077 11201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:39:57.933368 11201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:39:57.933725 11201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:39:57.933892 11201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:39:57.937956 11201 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.126:44439
I20260619 02:39:57.938020 11267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.126:44439 every 8 connection(s)
I20260619 02:39:57.938342 11201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:39:57.939364 11268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:39:57.941618 11268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d6aba8fe5394b6fb3053676ad7438c9: Bootstrap starting.
I20260619 02:39:57.942206 11268 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8d6aba8fe5394b6fb3053676ad7438c9: Neither blocks nor log segments found. Creating new log.
I20260619 02:39:57.942495 11268 log.cc:824] T 00000000000000000000000000000000 P 8d6aba8fe5394b6fb3053676ad7438c9: Log is configured to *not* fsync() on all Append() calls
I20260619 02:39:57.943284 11268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d6aba8fe5394b6fb3053676ad7438c9: No bootstrap required, opened a new log
I20260619 02:39:57.943516 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 11201
I20260619 02:39:57.943639 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836772852275-6085-0/minicluster-data/master-0/wal/instance
I20260619 02:39:57.943918 6085 external_mini_cluster.cc:1428] Setting key a7501e1f021d4aee97d09551a68dc13d
I20260619 02:39:57.945770 11268 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d6aba8fe5394b6fb3053676ad7438c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d6aba8fe5394b6fb3053676ad7438c9" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 44439 } }
I20260619 02:39:57.945950 11268 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8d6aba8fe5394b6fb3053676ad7438c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:39:57.945988 11268 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d6aba8fe5394b6fb3053676ad7438c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d6aba8fe5394b6fb3053676ad7438c9, State: Initialized, Role: FOLLOWER
I20260619 02:39:57.946146 11268 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d6aba8fe5394b6fb3053676ad7438c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d6aba8fe5394b6fb3053676ad7438c9" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 44439 } }
I20260619 02:39:57.946246 11268 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8d6aba8fe5394b6fb3053676ad7438c9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:39:57.946278 11268 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d6aba8fe5394b6fb3053676ad7438c9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:39:57.946316 11268 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d6aba8fe5394b6fb3053676ad7438c9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:39:57.947124 11268 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d6aba8fe5394b6fb3053676ad7438c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d6aba8fe5394b6fb3053676ad7438c9" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 44439 } }
I20260619 02:39:57.947237 11268 leader_election.cc:302] T 00000000000000000000000000000000 P 8d6aba8fe5394b6fb3053676ad7438c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8d6aba8fe5394b6fb3053676ad7438c9; no voters:
I20260619 02:39:57.947387 11268 leader_election.cc:288] T 00000000000000000000000000000000 P 8d6aba8fe5394b6fb3053676ad7438c9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:39:57.947510 11273 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8d6aba8fe5394b6fb3053676ad7438c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:39:57.947638 11268 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d6aba8fe5394b6fb3053676ad7438c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:39:57.947670 11273 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8d6aba8fe5394b6fb3053676ad7438c9 [term 1 LEADER]: Becoming Leader. State: Replica: 8d6aba8fe5394b6fb3053676ad7438c9, State: Running, Role: LEADER
I20260619 02:39:57.947793 11273 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8d6aba8fe5394b6fb3053676ad7438c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d6aba8fe5394b6fb3053676ad7438c9" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 44439 } }
I20260619 02:39:57.948170 11273 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d6aba8fe5394b6fb3053676ad7438c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d6aba8fe5394b6fb3053676ad7438c9. Latest consensus state: current_term: 1 leader_uuid: "8d6aba8fe5394b6fb3053676ad7438c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d6aba8fe5394b6fb3053676ad7438c9" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 44439 } } }
I20260619 02:39:57.948266 11273 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d6aba8fe5394b6fb3053676ad7438c9 [sys.catalog]: This master's current role is: LEADER
I20260619 02:39:57.948179 11275 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d6aba8fe5394b6fb3053676ad7438c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8d6aba8fe5394b6fb3053676ad7438c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d6aba8fe5394b6fb3053676ad7438c9" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 44439 } } }
I20260619 02:39:57.949496 11275 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d6aba8fe5394b6fb3053676ad7438c9 [sys.catalog]: This master's current role is: LEADER
W20260619 02:39:57.949843 11288 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8d6aba8fe5394b6fb3053676ad7438c9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:39:57.949898 11288 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:39:57.950098 11281 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:39:57.950273 11281 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:39:57.952044 11272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:57.944539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57206 (local address 127.5.241.126:44439)
0619 02:39:57.944744 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:57.944753 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:57.944789 (+ 36us) server_negotiation.cc:407] Connection header received
0619 02:39:57.945723 (+ 934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:57.945733 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:57.945819 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:57.945983 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:57.946271 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:57.948252 (+ 1981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:57.949553 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:57.950198 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:57.950321 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:57.951283 (+ 962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:57.951287 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:57.951288 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:39:57.951514 (+ 226us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:57.951554 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:39:57.951570 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:57.951665 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:57.951736 (+ 71us) server_negotiation.cc:299] Negotiation successful
0619 02:39:57.951817 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":78,"threads_started":1}
I20260619 02:39:57.953202 6085 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:39:57.958868 11281 catalog_manager.cc:1347] Generated new cluster ID: 801768aadf864771931ea63fee31247d
I20260619 02:39:57.958938 11281 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:39:57.993783 11281 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:39:57.994362 11281 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:39:58.003834 11281 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8d6aba8fe5394b6fb3053676ad7438c9: Generated new TSK 0
I20260619 02:39:58.004063 11281 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260619 02:39:58.038338 11298 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260619 02:39:58.038556 11298 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260619 02:39:58.039284 11297 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:39:58.021578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:44439 (local address 127.0.0.1:57212)
0619 02:39:58.021801 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:58.021811 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:58.022070 (+ 259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:58.022308 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:58.022309 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:58.022311 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0619 02:39:58.022376 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:58.022378 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:58.023323 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:58.023324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:58.023940 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:58.023942 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:58.024409 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:58.024410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:58.024480 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:58.024501 (+ 21us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 02:39:58.038762 (+ 14261us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37101/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0619 02:39:58.038904 (+ 142us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.241.126:44439: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37101/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":96,"threads_started":1}
I20260619 02:39:58.039633 11272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:58.021682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57212 (local address 127.5.241.126:44439)
0619 02:39:58.021739 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:58.021741 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:58.022030 (+ 289us) server_negotiation.cc:407] Connection header received
0619 02:39:58.022173 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:58.022175 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:58.022187 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:58.022265 (+ 78us) server_negotiation.cc:226] Negotiated authn=JWT
0619 02:39:58.022449 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:58.023229 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:58.024025 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:58.024307 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:58.024379 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:58.024625 (+ 246us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 02:39:58.038462 (+ 13837us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37101/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0619 02:39:58.039565 (+ 1103us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57212: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37101/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":26}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.5.241.126:44439: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37101/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
W20260619 02:39:58.039710 11272 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57212: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37101/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
I20260619 02:39:58.039747 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 11201
2026-06-19T02:39:58Z chronyd exiting
I20260619 02:39:58.076011 6085 test_util.cc:175] -----------------------------------------------
I20260619 02:39:58.076116 6085 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836772852275-6085-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (267 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260619 02:39:58.076886 6085 mini_oidc.cc:152] Starting JWKS server
I20260619 02:39:58.077577 6085 webserver.cc:460] Webserver started at https://127.0.0.1:45279/ using document root <none> and password file <none>
I20260619 02:39:58.077656 6085 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 02:39:58.077832 6085 webserver.cc:460] Webserver started at http://127.0.0.1:34649/ using document root <none> and password file <none>
2026-06-19T02:39:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:39:58Z Disabled control of system clock
I20260619 02:39:58.085880 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:46453
--webserver_interface=127.5.241.126
--webserver_port=0
--builtin_ntp_servers=127.5.241.84:39803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:46453
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45279/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836772852275-6085-0/testchainca.cert with env {}
W20260619 02:39:58.171012 11304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:39:58.171222 11304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:39:58.171245 11304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:39:58.172490 11304 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:39:58.172536 11304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:39:58.172549 11304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:39:58.172562 11304 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:39:58.172575 11304 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 02:39:58.172587 11304 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 02:39:58.172600 11304 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45279/jwks.json
I20260619 02:39:58.174151 11304 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:39803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836772852275-6085-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:46453
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:46453
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45279/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.241.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836772852275-6085-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11304
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:39:58.174412 11304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:39:58.174638 11304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:39:58.179709 11310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:58.179828 11312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:39:58.179949 11304 server_base.cc:1034] running on GCE node
W20260619 02:39:58.179970 11309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:39:58.180303 11304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:39:58.180619 11304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:39:58.181787 11304 hybrid_clock.cc:648] HybridClock initialized: now 1781836798181781 us; error 36 us; skew 500 ppm
I20260619 02:39:58.183151 11304 webserver.cc:460] Webserver started at http://127.5.241.126:43779/ using document root <none> and password file <none>
I20260619 02:39:58.183408 11304 fs_manager.cc:359] Metadata directory not provided
I20260619 02:39:58.183478 11304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:39:58.183645 11304 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:39:58.184718 11304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836772852275-6085-0/minicluster-data/master-0/data/instance:
uuid: "d9b13bfa9c874d0986c84f1e74c610c7"
format_stamp: "Formatted at 2026-06-19 02:39:58 on dist-test-slave-zpbg"
server_key: "52f666e4d4f49467640228cb0774e0c8"
server_key_iv: "40aa59f324736c369853c8ffb8c33d5c"
server_key_version: "encryptionkey@0"
I20260619 02:39:58.185081 11304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836772852275-6085-0/minicluster-data/master-0/wal/instance:
uuid: "d9b13bfa9c874d0986c84f1e74c610c7"
format_stamp: "Formatted at 2026-06-19 02:39:58 on dist-test-slave-zpbg"
server_key: "52f666e4d4f49467640228cb0774e0c8"
server_key_iv: "40aa59f324736c369853c8ffb8c33d5c"
server_key_version: "encryptionkey@0"
I20260619 02:39:58.186429 11304 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:39:58.187251 11318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:58.187423 11304 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:58.187529 11304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "d9b13bfa9c874d0986c84f1e74c610c7"
format_stamp: "Formatted at 2026-06-19 02:39:58 on dist-test-slave-zpbg"
server_key: "52f666e4d4f49467640228cb0774e0c8"
server_key_iv: "40aa59f324736c369853c8ffb8c33d5c"
server_key_version: "encryptionkey@0"
I20260619 02:39:58.187624 11304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:39:58.202186 11304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:39:58.202517 11304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:39:58.202679 11304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:39:58.206866 11304 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.126:46453
I20260619 02:39:58.206930 11370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.126:46453 every 8 connection(s)
I20260619 02:39:58.207291 11304 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:39:58.208259 11371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:39:58.210634 11371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9b13bfa9c874d0986c84f1e74c610c7: Bootstrap starting.
I20260619 02:39:58.210883 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 11304
I20260619 02:39:58.210994 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836772852275-6085-0/minicluster-data/master-0/wal/instance
I20260619 02:39:58.211201 6085 external_mini_cluster.cc:1428] Setting key 78dc4ccefedebe4d4e2802e12d5ecae2
I20260619 02:39:58.211413 11371 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d9b13bfa9c874d0986c84f1e74c610c7: Neither blocks nor log segments found. Creating new log.
I20260619 02:39:58.211747 11371 log.cc:824] T 00000000000000000000000000000000 P d9b13bfa9c874d0986c84f1e74c610c7: Log is configured to *not* fsync() on all Append() calls
I20260619 02:39:58.212420 11371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9b13bfa9c874d0986c84f1e74c610c7: No bootstrap required, opened a new log
I20260619 02:39:58.214200 11371 raft_consensus.cc:348] T 00000000000000000000000000000000 P d9b13bfa9c874d0986c84f1e74c610c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9b13bfa9c874d0986c84f1e74c610c7" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 46453 } }
I20260619 02:39:58.214365 11371 raft_consensus.cc:374] T 00000000000000000000000000000000 P d9b13bfa9c874d0986c84f1e74c610c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:39:58.214411 11371 raft_consensus.cc:729] T 00000000000000000000000000000000 P d9b13bfa9c874d0986c84f1e74c610c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9b13bfa9c874d0986c84f1e74c610c7, State: Initialized, Role: FOLLOWER
I20260619 02:39:58.214546 11371 consensus_queue.cc:260] T 00000000000000000000000000000000 P d9b13bfa9c874d0986c84f1e74c610c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9b13bfa9c874d0986c84f1e74c610c7" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 46453 } }
I20260619 02:39:58.214653 11371 raft_consensus.cc:388] T 00000000000000000000000000000000 P d9b13bfa9c874d0986c84f1e74c610c7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:39:58.214699 11371 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9b13bfa9c874d0986c84f1e74c610c7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:39:58.214751 11371 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d9b13bfa9c874d0986c84f1e74c610c7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:39:58.215384 11371 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9b13bfa9c874d0986c84f1e74c610c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9b13bfa9c874d0986c84f1e74c610c7" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 46453 } }
I20260619 02:39:58.215494 11371 leader_election.cc:302] T 00000000000000000000000000000000 P d9b13bfa9c874d0986c84f1e74c610c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d9b13bfa9c874d0986c84f1e74c610c7; no voters:
I20260619 02:39:58.215682 11371 leader_election.cc:288] T 00000000000000000000000000000000 P d9b13bfa9c874d0986c84f1e74c610c7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:39:58.215874 11377 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d9b13bfa9c874d0986c84f1e74c610c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:39:58.215978 11371 sys_catalog.cc:564] T 00000000000000000000000000000000 P d9b13bfa9c874d0986c84f1e74c610c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:39:58.216025 11377 raft_consensus.cc:686] T 00000000000000000000000000000000 P d9b13bfa9c874d0986c84f1e74c610c7 [term 1 LEADER]: Becoming Leader. State: Replica: d9b13bfa9c874d0986c84f1e74c610c7, State: Running, Role: LEADER
I20260619 02:39:58.216101 11377 consensus_queue.cc:237] T 00000000000000000000000000000000 P d9b13bfa9c874d0986c84f1e74c610c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9b13bfa9c874d0986c84f1e74c610c7" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 46453 } }
I20260619 02:39:58.216427 11377 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9b13bfa9c874d0986c84f1e74c610c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d9b13bfa9c874d0986c84f1e74c610c7. Latest consensus state: current_term: 1 leader_uuid: "d9b13bfa9c874d0986c84f1e74c610c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9b13bfa9c874d0986c84f1e74c610c7" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 46453 } } }
I20260619 02:39:58.216503 11377 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9b13bfa9c874d0986c84f1e74c610c7 [sys.catalog]: This master's current role is: LEADER
I20260619 02:39:58.216588 11376 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9b13bfa9c874d0986c84f1e74c610c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d9b13bfa9c874d0986c84f1e74c610c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9b13bfa9c874d0986c84f1e74c610c7" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 46453 } } }
I20260619 02:39:58.216847 11376 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9b13bfa9c874d0986c84f1e74c610c7 [sys.catalog]: This master's current role is: LEADER
I20260619 02:39:58.216974 11383 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:39:58.217022 11374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:58.211750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46780 (local address 127.5.241.126:46453)
0619 02:39:58.211978 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:58.211986 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:58.212020 (+ 34us) server_negotiation.cc:407] Connection header received
0619 02:39:58.212626 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:58.212633 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:58.212694 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:58.212844 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:58.213025 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:58.213912 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:58.214862 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:58.215279 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:58.215367 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:58.216209 (+ 842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:58.216213 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:58.216213 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:39:58.216422 (+ 209us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:58.216466 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:39:58.216481 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:58.216547 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:58.216659 (+ 112us) server_negotiation.cc:299] Negotiation successful
0619 02:39:58.216742 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":82,"threads_started":1}
I20260619 02:39:58.217423 11383 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:39:58.217967 6085 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:39:58.219384 11383 catalog_manager.cc:1347] Generated new cluster ID: 6239dabdb3c5413a819be147b8f1aa80
I20260619 02:39:58.219437 11383 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:39:58.246922 11383 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:39:58.247521 11383 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:39:58.262320 11383 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d9b13bfa9c874d0986c84f1e74c610c7: Generated new TSK 0
I20260619 02:39:58.262522 11383 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260619 02:39:58.289443 11401 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260619 02:39:58.289553 11401 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260619 02:39:58.289994 11374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:58.273681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46790 (local address 127.5.241.126:46453)
0619 02:39:58.273899 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:58.273901 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:58.274086 (+ 185us) server_negotiation.cc:407] Connection header received
0619 02:39:58.274129 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:58.274130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:58.274144 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:58.274222 (+ 78us) server_negotiation.cc:226] Negotiated authn=JWT
0619 02:39:58.274391 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:58.275262 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:58.276066 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:58.276382 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:58.276461 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:58.276609 (+ 148us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 02:39:58.289712 (+ 13103us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45279/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0619 02:39:58.289908 (+ 196us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46790: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45279/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":190}
W20260619 02:39:58.290058 11374 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46790: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45279/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260619 02:39:58.290073 11400 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:39:58.273613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:46453 (local address 127.0.0.1:46790)
0619 02:39:58.273961 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:58.273972 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:58.274040 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:58.274245 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:58.274246 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:58.274248 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0619 02:39:58.274302 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:58.274304 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:58.275373 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:58.275374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:58.275974 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:58.275976 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:58.276466 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:58.276466 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:58.276507 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:58.276511 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 02:39:58.289871 (+ 13360us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45279/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0619 02:39:58.289975 (+ 104us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.241.126:46453: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45279/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":259,"threads_started":1}
I20260619 02:39:58.290401 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 11304
2026-06-19T02:39:58Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (251 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260619 02:39:58.328608 6085 mini_oidc.cc:152] Starting JWKS server
I20260619 02:39:58.329316 6085 webserver.cc:460] Webserver started at https://127.0.0.1:40869/ using document root <none> and password file <none>
I20260619 02:39:58.329388 6085 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 02:39:58.329558 6085 webserver.cc:460] Webserver started at http://127.0.0.1:44173/ using document root <none> and password file <none>
2026-06-19T02:39:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:39:58Z Disabled control of system clock
I20260619 02:39:58.338716 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:44003
--webserver_interface=127.5.241.126
--webserver_port=0
--builtin_ntp_servers=127.5.241.84:35547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:44003
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40869/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260619 02:39:58.420493 11407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:39:58.420720 11407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:39:58.420743 11407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:39:58.421855 11407 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:39:58.421896 11407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:39:58.421911 11407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:39:58.421923 11407 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:39:58.421936 11407 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 02:39:58.421947 11407 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 02:39:58.421959 11407 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40869/jwks.json
I20260619 02:39:58.423280 11407 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:35547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781836772852275-6085-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:44003
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:44003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40869/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.241.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11407
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:39:58.423552 11407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:39:58.423815 11407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:39:58.428946 11407 server_base.cc:1034] running on GCE node
W20260619 02:39:58.428920 11412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:58.428916 11415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:58.428912 11413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:39:58.429445 11407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:39:58.429750 11407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:39:58.430938 11407 hybrid_clock.cc:648] HybridClock initialized: now 1781836798430914 us; error 44 us; skew 500 ppm
I20260619 02:39:58.432380 11407 webserver.cc:460] Webserver started at http://127.5.241.126:46127/ using document root <none> and password file <none>
I20260619 02:39:58.432649 11407 fs_manager.cc:359] Metadata directory not provided
I20260619 02:39:58.432729 11407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:39:58.432890 11407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:39:58.434037 11407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781836772852275-6085-0/minicluster-data/master-0/data/instance:
uuid: "fc73856733a94954b438cc96faf5d45a"
format_stamp: "Formatted at 2026-06-19 02:39:58 on dist-test-slave-zpbg"
server_key: "bee6de1ef4c1a7c3d8ace21037856276"
server_key_iv: "59288f098834bc71b0f58f90ca26793e"
server_key_version: "encryptionkey@0"
I20260619 02:39:58.434407 11407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781836772852275-6085-0/minicluster-data/master-0/wal/instance:
uuid: "fc73856733a94954b438cc96faf5d45a"
format_stamp: "Formatted at 2026-06-19 02:39:58 on dist-test-slave-zpbg"
server_key: "bee6de1ef4c1a7c3d8ace21037856276"
server_key_iv: "59288f098834bc71b0f58f90ca26793e"
server_key_version: "encryptionkey@0"
I20260619 02:39:58.435926 11407 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:39:58.436914 11421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:58.437083 11407 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:39:58.437194 11407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "fc73856733a94954b438cc96faf5d45a"
format_stamp: "Formatted at 2026-06-19 02:39:58 on dist-test-slave-zpbg"
server_key: "bee6de1ef4c1a7c3d8ace21037856276"
server_key_iv: "59288f098834bc71b0f58f90ca26793e"
server_key_version: "encryptionkey@0"
I20260619 02:39:58.437271 11407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:39:58.482888 11407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:39:58.483258 11407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:39:58.483426 11407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:39:58.487263 11407 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.126:44003
I20260619 02:39:58.487303 11473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.126:44003 every 8 connection(s)
I20260619 02:39:58.487679 11407 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:39:58.488343 11474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:39:58.489218 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 11407
I20260619 02:39:58.489346 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781836772852275-6085-0/minicluster-data/master-0/wal/instance
I20260619 02:39:58.489512 6085 external_mini_cluster.cc:1428] Setting key 94ccf434deeb8de9f286c83a1daf485c
I20260619 02:39:58.491717 11474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc73856733a94954b438cc96faf5d45a: Bootstrap starting.
I20260619 02:39:58.492434 11474 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc73856733a94954b438cc96faf5d45a: Neither blocks nor log segments found. Creating new log.
I20260619 02:39:58.492753 11474 log.cc:824] T 00000000000000000000000000000000 P fc73856733a94954b438cc96faf5d45a: Log is configured to *not* fsync() on all Append() calls
I20260619 02:39:58.493505 11474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc73856733a94954b438cc96faf5d45a: No bootstrap required, opened a new log
I20260619 02:39:58.495289 11477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:58.490085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46552 (local address 127.5.241.126:44003)
0619 02:39:58.490279 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:58.490286 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:58.490310 (+ 24us) server_negotiation.cc:407] Connection header received
0619 02:39:58.490717 (+ 407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:58.490724 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:58.490841 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:58.490960 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:58.491203 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:58.492117 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:58.493213 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:58.493678 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:58.493781 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:58.494520 (+ 739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:58.494523 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:58.494524 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:39:58.494694 (+ 170us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:58.494727 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:39:58.494739 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:58.494864 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:58.494973 (+ 109us) server_negotiation.cc:299] Negotiation successful
0619 02:39:58.495063 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":85,"threads_started":1}
I20260619 02:39:58.495977 11474 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc73856733a94954b438cc96faf5d45a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc73856733a94954b438cc96faf5d45a" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 44003 } }
I20260619 02:39:58.496157 11474 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc73856733a94954b438cc96faf5d45a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:39:58.496186 11474 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc73856733a94954b438cc96faf5d45a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc73856733a94954b438cc96faf5d45a, State: Initialized, Role: FOLLOWER
I20260619 02:39:58.496318 11474 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc73856733a94954b438cc96faf5d45a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc73856733a94954b438cc96faf5d45a" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 44003 } }
I20260619 02:39:58.496420 11474 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc73856733a94954b438cc96faf5d45a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:39:58.496459 11474 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc73856733a94954b438cc96faf5d45a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:39:58.496518 11474 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc73856733a94954b438cc96faf5d45a [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:39:58.496570 6085 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:39:58.497159 11474 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc73856733a94954b438cc96faf5d45a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc73856733a94954b438cc96faf5d45a" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 44003 } }
I20260619 02:39:58.497272 11474 leader_election.cc:302] T 00000000000000000000000000000000 P fc73856733a94954b438cc96faf5d45a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc73856733a94954b438cc96faf5d45a; no voters:
I20260619 02:39:58.497450 11474 leader_election.cc:288] T 00000000000000000000000000000000 P fc73856733a94954b438cc96faf5d45a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:39:58.497524 11479 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc73856733a94954b438cc96faf5d45a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:39:58.497771 11479 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc73856733a94954b438cc96faf5d45a [term 1 LEADER]: Becoming Leader. State: Replica: fc73856733a94954b438cc96faf5d45a, State: Running, Role: LEADER
I20260619 02:39:58.497807 11474 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc73856733a94954b438cc96faf5d45a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:39:58.497996 11479 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc73856733a94954b438cc96faf5d45a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc73856733a94954b438cc96faf5d45a" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 44003 } }
I20260619 02:39:58.498409 11481 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc73856733a94954b438cc96faf5d45a [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc73856733a94954b438cc96faf5d45a. Latest consensus state: current_term: 1 leader_uuid: "fc73856733a94954b438cc96faf5d45a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc73856733a94954b438cc96faf5d45a" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 44003 } } }
I20260619 02:39:58.498498 11481 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc73856733a94954b438cc96faf5d45a [sys.catalog]: This master's current role is: LEADER
I20260619 02:39:58.498910 11487 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:39:58.498929 11480 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc73856733a94954b438cc96faf5d45a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc73856733a94954b438cc96faf5d45a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc73856733a94954b438cc96faf5d45a" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 44003 } } }
I20260619 02:39:58.499046 11480 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc73856733a94954b438cc96faf5d45a [sys.catalog]: This master's current role is: LEADER
I20260619 02:39:58.499559 11487 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:39:58.501296 11487 catalog_manager.cc:1347] Generated new cluster ID: a4c1eb632736498da9cbd72661636475
I20260619 02:39:58.501338 11487 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:39:58.523878 11487 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:39:58.524425 11487 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:39:58.540472 11487 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc73856733a94954b438cc96faf5d45a: Generated new TSK 0
I20260619 02:39:58.540707 11487 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:39:58.555882 11477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:58.551966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46560 (local address 127.5.241.126:44003)
0619 02:39:58.552035 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:58.552037 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:58.552153 (+ 116us) server_negotiation.cc:407] Connection header received
0619 02:39:58.552229 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:58.552230 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:58.552241 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:58.552306 (+ 65us) server_negotiation.cc:226] Negotiated authn=JWT
0619 02:39:58.552447 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:58.553018 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:58.554143 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:58.554345 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:58.554405 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:58.554605 (+ 200us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 02:39:58.555659 (+ 1054us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40869/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0619 02:39:58.555802 (+ 143us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46560: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40869/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":42}
W20260619 02:39:58.555959 11477 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46560: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40869/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260619 02:39:58.555975 11503 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:39:58.551902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:44003 (local address 127.0.0.1:46560)
0619 02:39:58.552075 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:58.552087 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:58.552157 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:58.552320 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:58.552321 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:58.552323 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0619 02:39:58.552383 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:58.552385 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:58.553126 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:58.553127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:58.554054 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:58.554056 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:58.554434 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:58.554435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:58.554501 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:58.554507 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 02:39:58.555769 (+ 1262us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40869/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0619 02:39:58.555870 (+ 101us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.241.126:44003: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40869/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":111,"threads_started":1}
I20260619 02:39:58.556406 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 11407
E20260619 02:39:58.568574 11504 webserver.cc:533] Webserver: sslize failed:
E20260619 02:39:58.568648 11504 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-06-19T02:39:58Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (265 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-06-19T02:39:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:39:58Z Disabled control of system clock
I20260619 02:39:58.601619 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:45551
--webserver_interface=127.5.241.126
--webserver_port=0
--builtin_ntp_servers=127.5.241.84:38407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:45551
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836772852275-6085-0/testchainca.cert with env {}
W20260619 02:39:58.682061 11508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:39:58.682283 11508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:39:58.682307 11508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:39:58.683458 11508 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:39:58.683495 11508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:39:58.683508 11508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:39:58.683521 11508 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:39:58.683533 11508 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 02:39:58.683545 11508 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 02:39:58.683557 11508 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260619 02:39:58.684995 11508 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:38407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836772852275-6085-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:45551
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:45551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.241.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836772852275-6085-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11508
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:39:58.685245 11508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:39:58.685498 11508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:39:58.690569 11516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:58.690568 11513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:39:58.690565 11514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:39:58.690994 11508 server_base.cc:1034] running on GCE node
I20260619 02:39:58.691239 11508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:39:58.691540 11508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:39:58.692698 11508 hybrid_clock.cc:648] HybridClock initialized: now 1781836798692692 us; error 38 us; skew 500 ppm
I20260619 02:39:58.693919 11508 webserver.cc:460] Webserver started at http://127.5.241.126:39509/ using document root <none> and password file <none>
I20260619 02:39:58.694162 11508 fs_manager.cc:359] Metadata directory not provided
I20260619 02:39:58.694237 11508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:39:58.694391 11508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:39:58.695621 11508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836772852275-6085-0/minicluster-data/master-0/data/instance:
uuid: "a4489b78be924c61b42c478670fa3afa"
format_stamp: "Formatted at 2026-06-19 02:39:58 on dist-test-slave-zpbg"
server_key: "217a8caf76242aff2a54cacb8f96503c"
server_key_iv: "845cb7e79c5254a5cf4ac27fd719d8e8"
server_key_version: "encryptionkey@0"
I20260619 02:39:58.695976 11508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836772852275-6085-0/minicluster-data/master-0/wal/instance:
uuid: "a4489b78be924c61b42c478670fa3afa"
format_stamp: "Formatted at 2026-06-19 02:39:58 on dist-test-slave-zpbg"
server_key: "217a8caf76242aff2a54cacb8f96503c"
server_key_iv: "845cb7e79c5254a5cf4ac27fd719d8e8"
server_key_version: "encryptionkey@0"
I20260619 02:39:58.697281 11508 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 02:39:58.698004 11522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:39:58.698172 11508 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:39:58.698275 11508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "a4489b78be924c61b42c478670fa3afa"
format_stamp: "Formatted at 2026-06-19 02:39:58 on dist-test-slave-zpbg"
server_key: "217a8caf76242aff2a54cacb8f96503c"
server_key_iv: "845cb7e79c5254a5cf4ac27fd719d8e8"
server_key_version: "encryptionkey@0"
I20260619 02:39:58.698348 11508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:39:58.724325 11508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:39:58.724663 11508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:39:58.724812 11508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:39:58.728681 11508 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.126:45551
I20260619 02:39:58.728725 11574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.126:45551 every 8 connection(s)
I20260619 02:39:58.729096 11508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:39:58.729993 11575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:39:58.732326 11575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4489b78be924c61b42c478670fa3afa: Bootstrap starting.
I20260619 02:39:58.732889 11575 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a4489b78be924c61b42c478670fa3afa: Neither blocks nor log segments found. Creating new log.
I20260619 02:39:58.733177 11575 log.cc:824] T 00000000000000000000000000000000 P a4489b78be924c61b42c478670fa3afa: Log is configured to *not* fsync() on all Append() calls
I20260619 02:39:58.733892 11575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4489b78be924c61b42c478670fa3afa: No bootstrap required, opened a new log
I20260619 02:39:58.735566 11575 raft_consensus.cc:348] T 00000000000000000000000000000000 P a4489b78be924c61b42c478670fa3afa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4489b78be924c61b42c478670fa3afa" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45551 } }
I20260619 02:39:58.735704 11575 raft_consensus.cc:374] T 00000000000000000000000000000000 P a4489b78be924c61b42c478670fa3afa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:39:58.735726 11575 raft_consensus.cc:729] T 00000000000000000000000000000000 P a4489b78be924c61b42c478670fa3afa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4489b78be924c61b42c478670fa3afa, State: Initialized, Role: FOLLOWER
I20260619 02:39:58.735898 11575 consensus_queue.cc:260] T 00000000000000000000000000000000 P a4489b78be924c61b42c478670fa3afa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4489b78be924c61b42c478670fa3afa" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45551 } }
I20260619 02:39:58.735983 11575 raft_consensus.cc:388] T 00000000000000000000000000000000 P a4489b78be924c61b42c478670fa3afa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:39:58.736007 11575 raft_consensus.cc:482] T 00000000000000000000000000000000 P a4489b78be924c61b42c478670fa3afa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:39:58.736003 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 11508
I20260619 02:39:58.736074 11575 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a4489b78be924c61b42c478670fa3afa [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:39:58.736130 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836772852275-6085-0/minicluster-data/master-0/wal/instance
I20260619 02:39:58.736303 6085 external_mini_cluster.cc:1428] Setting key 0b50a6855c0e00d5007ee0e1a5bc7a16
I20260619 02:39:58.736805 11575 raft_consensus.cc:504] T 00000000000000000000000000000000 P a4489b78be924c61b42c478670fa3afa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4489b78be924c61b42c478670fa3afa" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45551 } }
I20260619 02:39:58.736909 11575 leader_election.cc:302] T 00000000000000000000000000000000 P a4489b78be924c61b42c478670fa3afa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a4489b78be924c61b42c478670fa3afa; no voters:
I20260619 02:39:58.737062 11575 leader_election.cc:288] T 00000000000000000000000000000000 P a4489b78be924c61b42c478670fa3afa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:39:58.737112 11580 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a4489b78be924c61b42c478670fa3afa [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:39:58.737282 11580 raft_consensus.cc:686] T 00000000000000000000000000000000 P a4489b78be924c61b42c478670fa3afa [term 1 LEADER]: Becoming Leader. State: Replica: a4489b78be924c61b42c478670fa3afa, State: Running, Role: LEADER
I20260619 02:39:58.737308 11575 sys_catalog.cc:564] T 00000000000000000000000000000000 P a4489b78be924c61b42c478670fa3afa [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:39:58.737376 11580 consensus_queue.cc:237] T 00000000000000000000000000000000 P a4489b78be924c61b42c478670fa3afa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4489b78be924c61b42c478670fa3afa" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45551 } }
I20260619 02:39:58.737695 11580 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4489b78be924c61b42c478670fa3afa [sys.catalog]: SysCatalogTable state changed. Reason: New leader a4489b78be924c61b42c478670fa3afa. Latest consensus state: current_term: 1 leader_uuid: "a4489b78be924c61b42c478670fa3afa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4489b78be924c61b42c478670fa3afa" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45551 } } }
I20260619 02:39:58.737785 11580 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4489b78be924c61b42c478670fa3afa [sys.catalog]: This master's current role is: LEADER
I20260619 02:39:58.737699 11582 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4489b78be924c61b42c478670fa3afa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a4489b78be924c61b42c478670fa3afa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4489b78be924c61b42c478670fa3afa" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45551 } } }
I20260619 02:39:58.737869 11582 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4489b78be924c61b42c478670fa3afa [sys.catalog]: This master's current role is: LEADER
I20260619 02:39:58.738003 11587 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:39:58.738498 11587 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:39:58.740046 11587 catalog_manager.cc:1347] Generated new cluster ID: 7b6184fb961444b6ac00b4db0e20921f
I20260619 02:39:58.740099 11587 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:39:58.742316 11579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:58.736808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41862 (local address 127.5.241.126:45551)
0619 02:39:58.737071 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:58.737081 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:58.737117 (+ 36us) server_negotiation.cc:407] Connection header received
0619 02:39:58.737237 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:58.737244 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:58.737312 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:58.737697 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:39:58.737803 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:58.739432 (+ 1629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:58.740416 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:58.740815 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:58.740941 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:58.741630 (+ 689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:39:58.741634 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:39:58.741635 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:39:58.741848 (+ 213us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:39:58.741888 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:39:58.741902 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:39:58.741972 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 02:39:58.742043 (+ 71us) server_negotiation.cc:299] Negotiation successful
0619 02:39:58.742118 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":88,"threads_started":1}
I20260619 02:39:58.743208 6085 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:39:58.754511 11587 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:39:58.755307 11587 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:39:58.757893 11587 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a4489b78be924c61b42c478670fa3afa: Generated new TSK 0
I20260619 02:39:58.758101 11587 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:39:59.242416 11579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:39:58.765012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41870 (local address 127.5.241.126:45551)
0619 02:39:58.765060 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 02:39:58.765062 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:39:58.765138 (+ 76us) server_negotiation.cc:407] Connection header received
0619 02:39:58.765224 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:39:58.765225 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:39:58.765234 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:39:58.765295 (+ 61us) server_negotiation.cc:226] Negotiated authn=JWT
0619 02:39:58.765425 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:58.765922 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:58.767041 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:39:58.767251 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:39:58.767308 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:58.767491 (+ 183us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 02:39:59.242196 (+474705us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0619 02:39:59.242355 (+ 159us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41870: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":26}
W20260619 02:39:59.242507 11579 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41870: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260619 02:39:59.242614 11604 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:39:58.764925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45551 (local address 127.0.0.1:41870)
0619 02:39:58.765079 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0619 02:39:58.765090 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:39:58.765156 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:39:58.765304 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:39:58.765305 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:39:58.765307 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0619 02:39:58.765362 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:58.765364 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:58.765997 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:58.765998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:58.766958 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:39:58.766962 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:39:58.767331 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:39:58.767332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:39:58.767392 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:39:58.767397 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 02:39:59.242350 (+474953us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0619 02:39:59.242501 (+ 151us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.241.126:45551: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":91,"threads_started":1}
I20260619 02:39:59.243141 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 11508
2026-06-19T02:39:59Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (666 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (87 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (89 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:39:59 dist-test-slave-zpbg krb5kdc[11616](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:39:59 dist-test-slave-zpbg krb5kdc[11616](info): set up 2 sockets
Jun 19 02:39:59 dist-test-slave-zpbg krb5kdc[11616](info): commencing operation
krb5kdc: starting...
W20260619 02:40:00.462996 6085 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Jun 19 02:40:00 dist-test-slave-zpbg krb5kdc[11616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836800, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:40:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:40:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.241.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
I20260619 02:40:00.582333 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:35633
--webserver_interface=127.5.241.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:45377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:35633
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:40:00.663488 11629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:00.663707 11629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:00.663729 11629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:00.664949 11629 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:40:00.664996 11629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:00.665010 11629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:00.665023 11629 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:40:00.665035 11629 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:40:00.666565 11629 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:45377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:35633
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:35633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.241.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11629
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:40:00.666863 11629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:00.667124 11629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:00.672930 11635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:00.673022 11629 server_base.cc:1034] running on GCE node
W20260619 02:40:00.672886 11637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:00.672976 11634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:00.673413 11629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:00.673760 11629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:00.674938 11629 hybrid_clock.cc:648] HybridClock initialized: now 1781836800674906 us; error 55 us; skew 500 ppm
Jun 19 02:40:00 dist-test-slave-zpbg krb5kdc[11616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836800, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:00.677901 11629 init.cc:377] Logged in from keytab as kudu/127.5.241.126@KRBTEST.COM (short username kudu)
I20260619 02:40:00.678927 11629 webserver.cc:460] Webserver started at http://127.5.241.126:41021/ using document root <none> and password file <none>
I20260619 02:40:00.679224 11629 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:00.679298 11629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:00.679461 11629 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:40:00.680707 11629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/master-0/data/instance:
uuid: "a9501faeddc24874839e5478fe78ee3f"
format_stamp: "Formatted at 2026-06-19 02:40:00 on dist-test-slave-zpbg"
server_key: "817c2856db8f586af39f896c39929f05"
server_key_iv: "450c8fb1b56ea75dd524a0e72dc8df6c"
server_key_version: "encryptionkey@0"
I20260619 02:40:00.681141 11629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/master-0/wal/instance:
uuid: "a9501faeddc24874839e5478fe78ee3f"
format_stamp: "Formatted at 2026-06-19 02:40:00 on dist-test-slave-zpbg"
server_key: "817c2856db8f586af39f896c39929f05"
server_key_iv: "450c8fb1b56ea75dd524a0e72dc8df6c"
server_key_version: "encryptionkey@0"
I20260619 02:40:00.682601 11629 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:40:00.683625 11644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:00.684034 11629 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:40:00.684183 11629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "a9501faeddc24874839e5478fe78ee3f"
format_stamp: "Formatted at 2026-06-19 02:40:00 on dist-test-slave-zpbg"
server_key: "817c2856db8f586af39f896c39929f05"
server_key_iv: "450c8fb1b56ea75dd524a0e72dc8df6c"
server_key_version: "encryptionkey@0"
I20260619 02:40:00.684306 11629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:00.707185 11629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:00.709688 11629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:00.709863 11629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:00.714133 11629 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.126:35633
I20260619 02:40:00.714166 11696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.126:35633 every 8 connection(s)
I20260619 02:40:00.714540 11629 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:40:00.715430 11697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:00.717558 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 11629
I20260619 02:40:00.717665 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/master-0/wal/instance
I20260619 02:40:00.717819 11697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9501faeddc24874839e5478fe78ee3f: Bootstrap starting.
I20260619 02:40:00.717860 6085 external_mini_cluster.cc:1428] Setting key ab56027cf1a57240d9b5a34613b8b52f
I20260619 02:40:00.718483 11697 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9501faeddc24874839e5478fe78ee3f: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:00.718828 11697 log.cc:824] T 00000000000000000000000000000000 P a9501faeddc24874839e5478fe78ee3f: Log is configured to *not* fsync() on all Append() calls
I20260619 02:40:00.719568 11697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9501faeddc24874839e5478fe78ee3f: No bootstrap required, opened a new log
I20260619 02:40:00.721730 11697 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9501faeddc24874839e5478fe78ee3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9501faeddc24874839e5478fe78ee3f" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 35633 } }
I20260619 02:40:00.721910 11697 raft_consensus.cc:374] T 00000000000000000000000000000000 P a9501faeddc24874839e5478fe78ee3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:00.721941 11697 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9501faeddc24874839e5478fe78ee3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9501faeddc24874839e5478fe78ee3f, State: Initialized, Role: FOLLOWER
I20260619 02:40:00.722054 11697 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9501faeddc24874839e5478fe78ee3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9501faeddc24874839e5478fe78ee3f" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 35633 } }
I20260619 02:40:00.722122 11697 raft_consensus.cc:388] T 00000000000000000000000000000000 P a9501faeddc24874839e5478fe78ee3f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:40:00.722155 11697 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9501faeddc24874839e5478fe78ee3f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:40:00.722210 11697 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9501faeddc24874839e5478fe78ee3f [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:00.723068 11697 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9501faeddc24874839e5478fe78ee3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9501faeddc24874839e5478fe78ee3f" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 35633 } }
I20260619 02:40:00.723186 11697 leader_election.cc:302] T 00000000000000000000000000000000 P a9501faeddc24874839e5478fe78ee3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a9501faeddc24874839e5478fe78ee3f; no voters:
I20260619 02:40:00.723348 11697 leader_election.cc:288] T 00000000000000000000000000000000 P a9501faeddc24874839e5478fe78ee3f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:40:00.723392 11702 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9501faeddc24874839e5478fe78ee3f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:40:00.723587 11702 raft_consensus.cc:686] T 00000000000000000000000000000000 P a9501faeddc24874839e5478fe78ee3f [term 1 LEADER]: Becoming Leader. State: Replica: a9501faeddc24874839e5478fe78ee3f, State: Running, Role: LEADER
I20260619 02:40:00.723613 11697 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9501faeddc24874839e5478fe78ee3f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:40:00.723686 11702 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9501faeddc24874839e5478fe78ee3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9501faeddc24874839e5478fe78ee3f" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 35633 } }
I20260619 02:40:00.724002 11703 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9501faeddc24874839e5478fe78ee3f [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9501faeddc24874839e5478fe78ee3f. Latest consensus state: current_term: 1 leader_uuid: "a9501faeddc24874839e5478fe78ee3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9501faeddc24874839e5478fe78ee3f" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 35633 } } }
I20260619 02:40:00.724088 11703 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9501faeddc24874839e5478fe78ee3f [sys.catalog]: This master's current role is: LEADER
I20260619 02:40:00.724319 11708 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:40:00.724429 11704 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9501faeddc24874839e5478fe78ee3f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a9501faeddc24874839e5478fe78ee3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9501faeddc24874839e5478fe78ee3f" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 35633 } } }
I20260619 02:40:00.724541 11704 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9501faeddc24874839e5478fe78ee3f [sys.catalog]: This master's current role is: LEADER
I20260619 02:40:00.724855 11708 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 19 02:40:00 dist-test-slave-zpbg krb5kdc[11616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836800, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:40:00.726554 11708 catalog_manager.cc:1347] Generated new cluster ID: a1ae2e89e5864b26a9c6f759cbb66441
I20260619 02:40:00.726605 11708 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:40:00.738502 11700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:00.718413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49452 (local address 127.5.241.126:35633)
0619 02:40:00.718649 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:00.718658 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:00.718698 (+ 40us) server_negotiation.cc:407] Connection header received
0619 02:40:00.719248 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:00.719255 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:00.719316 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:00.719439 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:00.720278 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:00.721146 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:00.722245 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:00.722988 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:00.723104 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:00.726202 (+ 3098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:00.726207 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:00.726207 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:00.726219 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:00.736688 (+ 10469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:00.737149 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:00.737153 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:00.737153 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:00.737262 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:00.737487 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:00.737488 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:00.737489 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:00.737819 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:00.737952 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:00.738127 (+ 175us) server_negotiation.cc:299] Negotiation successful
0619 02:40:00.738266 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":83,"threads_started":1}
I20260619 02:40:00.744033 11708 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:40:00.744712 11708 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:40:00.753120 11708 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a9501faeddc24874839e5478fe78ee3f: Generated new TSK 0
I20260619 02:40:00.753351 11708 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.241.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab.
Entry for principal kudu/127.5.241.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab.
I20260619 02:40:00.787036 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.65:0
--local_ip_for_outbound_sockets=127.5.241.65
--webserver_interface=127.5.241.65
--webserver_port=0
--tserver_master_addrs=127.5.241.126:35633
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab
--principal=kudu/127.5.241.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:45377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:40:00.867532 11723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:00.867729 11723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:00.867753 11723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:00.868817 11723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:00.868849 11723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:00.868875 11723 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.65
I20260619 02:40:00.870239 11723 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:45377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab
--principal=kudu/127.5.241.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.241.65
--webserver_port=0
--tserver_master_addrs=127.5.241.126:35633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11723
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.65
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:40:00.870414 11723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:00.870574 11723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:00.875478 11728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:00.875478 11731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:00.875593 11729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:00.875599 11723 server_base.cc:1034] running on GCE node
I20260619 02:40:00.875945 11723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:00.876194 11723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:00.877338 11723 hybrid_clock.cc:648] HybridClock initialized: now 1781836800877329 us; error 28 us; skew 500 ppm
Jun 19 02:40:00 dist-test-slave-zpbg krb5kdc[11616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836800, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:00.879859 11723 init.cc:377] Logged in from keytab as kudu/127.5.241.65@KRBTEST.COM (short username kudu)
I20260619 02:40:00.880275 11723 webserver.cc:460] Webserver started at http://127.5.241.65:35979/ using document root <none> and password file <none>
I20260619 02:40:00.880509 11723 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:00.880577 11723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:00.880722 11723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:40:00.881735 11723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/ts-0/data/instance:
uuid: "bbbeef191c014a1fad47d0661a524915"
format_stamp: "Formatted at 2026-06-19 02:40:00 on dist-test-slave-zpbg"
server_key: "dfdff0c80629eb28ee2e616c785e39de"
server_key_iv: "bca5fddafa7e858826de0a223e654a21"
server_key_version: "encryptionkey@0"
I20260619 02:40:00.882090 11723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/ts-0/wal/instance:
uuid: "bbbeef191c014a1fad47d0661a524915"
format_stamp: "Formatted at 2026-06-19 02:40:00 on dist-test-slave-zpbg"
server_key: "dfdff0c80629eb28ee2e616c785e39de"
server_key_iv: "bca5fddafa7e858826de0a223e654a21"
server_key_version: "encryptionkey@0"
I20260619 02:40:00.883446 11723 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:40:00.884183 11738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:00.884361 11723 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:40:00.884459 11723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/ts-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/ts-0/wal
uuid: "bbbeef191c014a1fad47d0661a524915"
format_stamp: "Formatted at 2026-06-19 02:40:00 on dist-test-slave-zpbg"
server_key: "dfdff0c80629eb28ee2e616c785e39de"
server_key_iv: "bca5fddafa7e858826de0a223e654a21"
server_key_version: "encryptionkey@0"
I20260619 02:40:00.884536 11723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:00.910881 11723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:00.912946 11723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:00.913156 11723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:00.913408 11723 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:40:00.913717 11723 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:40:00.913769 11723 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:00.913811 11723 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:40:00.913861 11723 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:00.919579 11723 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.65:43851
I20260619 02:40:00.919664 11851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.65:43851 every 8 connection(s)
I20260619 02:40:00.919996 11723 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
I20260619 02:40:00.922004 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 11723
I20260619 02:40:00.922171 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/ts-0/wal/instance
I20260619 02:40:00.922364 6085 external_mini_cluster.cc:1428] Setting key f5f5dae22c03c102c4044b46527413f4
Jun 19 02:40:00 dist-test-slave-zpbg krb5kdc[11616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836800, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.65@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.241.66@KRBTEST.COM; defaulting to no policy
I20260619 02:40:00.930590 11700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:00.920765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:50983 (local address 127.5.241.126:35633)
0619 02:40:00.920852 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:00.920855 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:00.921038 (+ 183us) server_negotiation.cc:407] Connection header received
0619 02:40:00.922030 (+ 992us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:00.922031 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:00.922044 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:00.922100 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:00.922845 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:00.923621 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:00.924970 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:00.925212 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:00.925290 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:00.927762 (+ 2472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:00.927764 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:00.927764 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:00.927772 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:00.929172 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:00.929619 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:00.929621 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:00.929621 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:00.929683 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:00.929942 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:00.929943 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:00.929944 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:00.930058 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:00.930133 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:00.930407 (+ 274us) server_negotiation.cc:299] Negotiation successful
0619 02:40:00.930494 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 02:40:00.930886 11854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:00.920738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:35633 (local address 127.5.241.65:50983)
0619 02:40:00.920916 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:00.920951 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:00.921913 (+ 962us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:00.922186 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:00.922190 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:00.922445 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:00.922633 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:00.922637 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:00.923741 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:00.923743 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:00.924893 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:00.924895 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:00.925300 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:00.925301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:00.925463 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:00.925987 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:00.925999 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:00.927651 (+ 1652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:00.929290 (+ 1639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:00.929297 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:00.929299 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:00.929528 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:00.929757 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:00.929758 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:00.929758 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:00.929872 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:00.930124 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:00.930130 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:00.930277 (+ 147us) client_negotiation.cc:769] Sending connection context
0619 02:40:00.930388 (+ 111us) client_negotiation.cc:240] Negotiation successful
0619 02:40:00.930552 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":84,"threads_started":1}
I20260619 02:40:00.931357 11852 heartbeater.cc:347] Connected to a master server at 127.5.241.126:35633
I20260619 02:40:00.931479 11852 heartbeater.cc:464] Registering TS with master...
I20260619 02:40:00.931666 11852 heartbeater.cc:511] Master 127.5.241.126:35633 requested a full tablet report, sending...
I20260619 02:40:00.932186 11661 ts_manager.cc:194] Registered new tserver with Master: bbbeef191c014a1fad47d0661a524915 (127.5.241.65:43851)
I20260619 02:40:00.932948 11661 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.65@KRBTEST.COM'} at 127.5.241.65:50983
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab.
Entry for principal kudu/127.5.241.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab.
I20260619 02:40:00.949966 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.66:0
--local_ip_for_outbound_sockets=127.5.241.66
--webserver_interface=127.5.241.66
--webserver_port=0
--tserver_master_addrs=127.5.241.126:35633
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab
--principal=kudu/127.5.241.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:45377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:40:01.030696 11857 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:01.031001 11857 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:01.031040 11857 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:01.032254 11857 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:01.032325 11857 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:01.032372 11857 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.66
I20260619 02:40:01.033903 11857 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:45377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab
--principal=kudu/127.5.241.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.241.66
--webserver_port=0
--tserver_master_addrs=127.5.241.126:35633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11857
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.66
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:40:01.034190 11857 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:01.034425 11857 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:01.039438 11862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:01.039461 11865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:01.039435 11863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:01.039837 11857 server_base.cc:1034] running on GCE node
I20260619 02:40:01.040061 11857 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:01.040259 11857 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:01.041394 11857 hybrid_clock.cc:648] HybridClock initialized: now 1781836801041386 us; error 27 us; skew 500 ppm
Jun 19 02:40:01 dist-test-slave-zpbg krb5kdc[11616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836801, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:01.043859 11857 init.cc:377] Logged in from keytab as kudu/127.5.241.66@KRBTEST.COM (short username kudu)
I20260619 02:40:01.044240 11857 webserver.cc:460] Webserver started at http://127.5.241.66:37191/ using document root <none> and password file <none>
I20260619 02:40:01.044488 11857 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:01.044533 11857 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:01.044618 11857 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:40:01.045603 11857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/ts-1/data/instance:
uuid: "51c2be8fdf704aa8b817bdb90d34d61f"
format_stamp: "Formatted at 2026-06-19 02:40:01 on dist-test-slave-zpbg"
server_key: "26f926a46313d90d86f1a91b15332dd1"
server_key_iv: "2893b24a6a33805c2dd4f5e2f1e851db"
server_key_version: "encryptionkey@0"
I20260619 02:40:01.045902 11857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/ts-1/wal/instance:
uuid: "51c2be8fdf704aa8b817bdb90d34d61f"
format_stamp: "Formatted at 2026-06-19 02:40:01 on dist-test-slave-zpbg"
server_key: "26f926a46313d90d86f1a91b15332dd1"
server_key_iv: "2893b24a6a33805c2dd4f5e2f1e851db"
server_key_version: "encryptionkey@0"
I20260619 02:40:01.047220 11857 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260619 02:40:01.047943 11872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:01.048178 11857 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:40:01.048264 11857 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/ts-1/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/ts-1/wal
uuid: "51c2be8fdf704aa8b817bdb90d34d61f"
format_stamp: "Formatted at 2026-06-19 02:40:01 on dist-test-slave-zpbg"
server_key: "26f926a46313d90d86f1a91b15332dd1"
server_key_iv: "2893b24a6a33805c2dd4f5e2f1e851db"
server_key_version: "encryptionkey@0"
I20260619 02:40:01.048355 11857 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:01.083299 11857 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:01.085640 11857 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:01.085810 11857 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:01.086054 11857 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:40:01.086364 11857 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:40:01.086397 11857 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:01.086421 11857 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:40:01.086460 11857 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:01.092146 11857 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.66:33783
I20260619 02:40:01.092231 11985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.66:33783 every 8 connection(s)
I20260619 02:40:01.092531 11857 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
I20260619 02:40:01.095042 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 11857
I20260619 02:40:01.095131 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/ts-1/wal/instance
I20260619 02:40:01.095309 6085 external_mini_cluster.cc:1428] Setting key 0cd30c8e4939f327acdb83313f1907fb
Jun 19 02:40:01 dist-test-slave-zpbg krb5kdc[11616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836801, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.66@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:40:01.102849 11700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:01.093256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:55151 (local address 127.5.241.126:35633)
0619 02:40:01.093321 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:01.093323 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:01.093524 (+ 201us) server_negotiation.cc:407] Connection header received
0619 02:40:01.094488 (+ 964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:01.094489 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:01.094501 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:01.094558 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:01.095302 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:01.095926 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:01.096942 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:01.097221 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:01.097309 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:01.099878 (+ 2569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:01.099879 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:01.099879 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:01.099886 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:01.101334 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:01.101829 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:01.101830 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:01.101830 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:01.101887 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:01.102157 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:01.102158 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:01.102158 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:01.102275 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:01.102326 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:01.102652 (+ 326us) server_negotiation.cc:299] Negotiation successful
0619 02:40:01.102723 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 02:40:01.103137 11988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:01.093226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:35633 (local address 127.5.241.66:55151)
0619 02:40:01.093403 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:01.093438 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:01.094376 (+ 938us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:01.094644 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:01.094647 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:01.094940 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:01.095181 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:01.095187 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:01.096026 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:01.096027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:01.096843 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:01.096845 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:01.097314 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:01.097315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:01.097428 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:01.097964 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:01.097973 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:01.099730 (+ 1757us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:01.101419 (+ 1689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:01.101424 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:01.101427 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:01.101737 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:01.101949 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:01.101950 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:01.101951 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:01.102091 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:01.102341 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:01.102347 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:01.102515 (+ 168us) client_negotiation.cc:769] Sending connection context
0619 02:40:01.102642 (+ 127us) client_negotiation.cc:240] Negotiation successful
0619 02:40:01.102854 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":86,"threads_started":1}
WARNING: no policy specified for kudu/127.5.241.67@KRBTEST.COM; defaulting to no policy
I20260619 02:40:01.103673 11986 heartbeater.cc:347] Connected to a master server at 127.5.241.126:35633
I20260619 02:40:01.103775 11986 heartbeater.cc:464] Registering TS with master...
I20260619 02:40:01.103957 11986 heartbeater.cc:511] Master 127.5.241.126:35633 requested a full tablet report, sending...
I20260619 02:40:01.104286 11661 ts_manager.cc:194] Registered new tserver with Master: 51c2be8fdf704aa8b817bdb90d34d61f (127.5.241.66:33783)
I20260619 02:40:01.104658 11661 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.66@KRBTEST.COM'} at 127.5.241.66:55151
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab.
Entry for principal kudu/127.5.241.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab.
I20260619 02:40:01.122759 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.67:0
--local_ip_for_outbound_sockets=127.5.241.67
--webserver_interface=127.5.241.67
--webserver_port=0
--tserver_master_addrs=127.5.241.126:35633
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab
--principal=kudu/127.5.241.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:45377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:40:01.202591 11991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:01.202910 11991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:01.202953 11991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:01.204144 11991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:01.204217 11991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:01.204264 11991 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.67
I20260619 02:40:01.205765 11991 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:45377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab
--principal=kudu/127.5.241.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.241.67
--webserver_port=0
--tserver_master_addrs=127.5.241.126:35633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.67
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:40:01.206033 11991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:01.206274 11991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:40:01.211680 11991 server_base.cc:1034] running on GCE node
W20260619 02:40:01.211709 11996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:01.211861 11999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:01.211679 11997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:01.212122 11991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:01.212368 11991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:01.213505 11991 hybrid_clock.cc:648] HybridClock initialized: now 1781836801213493 us; error 29 us; skew 500 ppm
Jun 19 02:40:01 dist-test-slave-zpbg krb5kdc[11616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836801, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:01.216048 11991 init.cc:377] Logged in from keytab as kudu/127.5.241.67@KRBTEST.COM (short username kudu)
I20260619 02:40:01.216480 11991 webserver.cc:460] Webserver started at http://127.5.241.67:32895/ using document root <none> and password file <none>
I20260619 02:40:01.216704 11991 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:01.216770 11991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:01.216917 11991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:40:01.217970 11991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/ts-2/data/instance:
uuid: "1f329cd087f84ae586a56394be007966"
format_stamp: "Formatted at 2026-06-19 02:40:01 on dist-test-slave-zpbg"
server_key: "4f048d89991597cbeb18a095677a9718"
server_key_iv: "7359fd7c31e27ffa903d09f4357aa9f7"
server_key_version: "encryptionkey@0"
I20260619 02:40:01.218343 11991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/ts-2/wal/instance:
uuid: "1f329cd087f84ae586a56394be007966"
format_stamp: "Formatted at 2026-06-19 02:40:01 on dist-test-slave-zpbg"
server_key: "4f048d89991597cbeb18a095677a9718"
server_key_iv: "7359fd7c31e27ffa903d09f4357aa9f7"
server_key_version: "encryptionkey@0"
I20260619 02:40:01.219729 11991 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260619 02:40:01.220566 12006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:01.220877 11991 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:40:01.220978 11991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/ts-2/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/ts-2/wal
uuid: "1f329cd087f84ae586a56394be007966"
format_stamp: "Formatted at 2026-06-19 02:40:01 on dist-test-slave-zpbg"
server_key: "4f048d89991597cbeb18a095677a9718"
server_key_iv: "7359fd7c31e27ffa903d09f4357aa9f7"
server_key_version: "encryptionkey@0"
I20260619 02:40:01.221050 11991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:01.232364 11991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:01.234704 11991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:01.234884 11991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:01.235121 11991 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:40:01.235421 11991 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:40:01.235473 11991 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:01.235515 11991 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:40:01.235541 11991 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:01.241607 11991 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.67:40265
I20260619 02:40:01.241688 12119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.67:40265 every 8 connection(s)
I20260619 02:40:01.242005 11991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
I20260619 02:40:01.247781 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 11991
I20260619 02:40:01.247902 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/minicluster-data/ts-2/wal/instance
I20260619 02:40:01.248140 6085 external_mini_cluster.cc:1428] Setting key 652ea7a3b33fbde1c1328abf4d50bd32
Jun 19 02:40:01 dist-test-slave-zpbg krb5kdc[11616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836801, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.67@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:40:01.251993 11700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:01.242755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:39025 (local address 127.5.241.126:35633)
0619 02:40:01.242879 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:01.242882 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:01.243138 (+ 256us) server_negotiation.cc:407] Connection header received
0619 02:40:01.244101 (+ 963us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:01.244102 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:01.244114 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:01.244172 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:01.244818 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:01.245314 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:01.246319 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:01.246516 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:01.246600 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:01.249112 (+ 2512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:01.249113 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:01.249113 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:01.249121 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:01.250544 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:01.251054 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:01.251055 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:01.251055 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:01.251122 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:01.251350 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:01.251351 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:01.251351 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:01.251458 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:01.251506 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:01.251845 (+ 339us) server_negotiation.cc:299] Negotiation successful
0619 02:40:01.251925 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260619 02:40:01.252307 12122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:01.242715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:35633 (local address 127.5.241.67:39025)
0619 02:40:01.242928 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:01.243032 (+ 104us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:01.243989 (+ 957us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:01.244256 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:01.244260 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:01.244528 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:01.244716 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:01.244721 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:01.245425 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:01.245426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:01.246209 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:01.246210 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:01.246609 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:01.246610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:01.246734 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:01.247314 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:01.247322 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:01.249034 (+ 1712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:01.250634 (+ 1600us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:01.250640 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:01.250642 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:01.250971 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:01.251186 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:01.251187 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:01.251187 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:01.251306 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:01.251528 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:01.251534 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:01.251667 (+ 133us) client_negotiation.cc:769] Sending connection context
0619 02:40:01.251762 (+ 95us) client_negotiation.cc:240] Negotiation successful
0619 02:40:01.251994 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":115,"threads_started":1}
I20260619 02:40:01.252800 12120 heartbeater.cc:347] Connected to a master server at 127.5.241.126:35633
I20260619 02:40:01.252943 12120 heartbeater.cc:464] Registering TS with master...
I20260619 02:40:01.253124 12120 heartbeater.cc:511] Master 127.5.241.126:35633 requested a full tablet report, sending...
I20260619 02:40:01.253515 11661 ts_manager.cc:194] Registered new tserver with Master: 1f329cd087f84ae586a56394be007966 (127.5.241.67:40265)
I20260619 02:40:01.253913 11661 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.67@KRBTEST.COM'} at 127.5.241.67:39025
I20260619 02:40:01.260528 6085 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/krb5kdc/test-admin.keytab.
Jun 19 02:40:01 dist-test-slave-zpbg krb5kdc[11616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836801, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:01.275791 6085 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260619 02:40:01.277717 12127 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:01.276755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48950 (local address 127.5.241.65:43851)
0619 02:40:01.276962 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:01.276966 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:01.276979 (+ 13us) server_negotiation.cc:407] Connection header received
0619 02:40:01.277007 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:01.277009 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:01.277024 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:01.277193 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:01.277619 (+ 426us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48950: BlockingRecv error: recv got EOF from 127.0.0.1:48950 (error 108)
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":79,"threads_started":1}
W20260619 02:40:01.277853 6085 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/krb5kdc/krb5cc' of type FILE
Jun 19 02:40:01 dist-test-slave-zpbg krb5kdc[11616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836801, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:01.278995 6085 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 02:40:01 dist-test-slave-zpbg krb5kdc[11616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836801, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:01.280356 6085 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 02:40:01 dist-test-slave-zpbg krb5kdc[11616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836801, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.241.65@KRBTEST.COM
I20260619 02:40:01.288955 12127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:01.280966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48954 (local address 127.5.241.65:43851)
0619 02:40:01.281034 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:01.281036 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:01.281100 (+ 64us) server_negotiation.cc:407] Connection header received
0619 02:40:01.281172 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:01.281173 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:01.281191 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:01.281257 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:01.281679 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:01.282424 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:01.283236 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:01.283558 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:01.283650 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:01.285882 (+ 2232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:01.285891 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:01.285891 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:01.285906 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:01.287786 (+ 1880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:01.288156 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:01.288160 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:01.288160 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:01.288228 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:01.288449 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:01.288450 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:01.288450 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:01.288612 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:01.288770 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:01.288812 (+ 42us) server_negotiation.cc:299] Negotiation successful
0619 02:40:01.288878 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260619 02:40:01.290918 12127 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:01.290307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48962 (local address 127.5.241.65:43851)
0619 02:40:01.290344 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:01.290346 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:01.290372 (+ 26us) server_negotiation.cc:407] Connection header received
0619 02:40:01.290402 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:01.290403 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:01.290412 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:01.290453 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:01.290862 (+ 409us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48962: BlockingRecv error: recv got EOF from 127.0.0.1:48962 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260619 02:40:01.291127 6085 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/krb5kdc/krb5cc' of type FILE
Jun 19 02:40:01 dist-test-slave-zpbg krb5kdc[11616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836801, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:01.292182 6085 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 02:40:01 dist-test-slave-zpbg krb5kdc[11616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836801, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:01.293031 6085 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 02:40:01 dist-test-slave-zpbg krb5kdc[11616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836801, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.241.65@KRBTEST.COM
I20260619 02:40:01.299942 12127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:01.293500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48966 (local address 127.5.241.65:43851)
0619 02:40:01.293581 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:01.293582 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:01.293636 (+ 54us) server_negotiation.cc:407] Connection header received
0619 02:40:01.293676 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:01.293677 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:01.293684 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:01.293731 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:01.294069 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:01.294562 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:01.295264 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:01.295479 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:01.295533 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:01.297458 (+ 1925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:01.297459 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:01.297460 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:01.297466 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:01.298874 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:01.299156 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:01.299157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:01.299157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:01.299204 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:01.299392 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:01.299393 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:01.299393 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:01.299515 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:01.299622 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:01.299696 (+ 74us) server_negotiation.cc:299] Negotiation successful
0619 02:40:01.299760 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260619 02:40:01.301661 12127 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:01.301076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48976 (local address 127.5.241.65:43851)
0619 02:40:01.301143 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:01.301145 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:01.301192 (+ 47us) server_negotiation.cc:407] Connection header received
0619 02:40:01.301252 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:01.301253 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:01.301260 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:01.301299 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:01.301600 (+ 301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48976: BlockingRecv error: recv got EOF from 127.0.0.1:48976 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260619 02:40:01.301702 6085 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836772852275-6085-0/krb5kdc/krb5cc' of type FILE
Jun 19 02:40:01 dist-test-slave-zpbg krb5kdc[11616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836801, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:01.302650 6085 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 02:40:01 dist-test-slave-zpbg krb5kdc[11616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836801, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:01.303508 6085 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 02:40:01 dist-test-slave-zpbg krb5kdc[11616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836801, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.241.65@KRBTEST.COM
I20260619 02:40:01.310140 12127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:01.303993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48980 (local address 127.5.241.65:43851)
0619 02:40:01.304062 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:01.304063 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:01.304084 (+ 21us) server_negotiation.cc:407] Connection header received
0619 02:40:01.304144 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:01.304145 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:01.304152 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:01.304198 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:01.304532 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:01.304976 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:01.305652 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:01.305835 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:01.305884 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:01.307809 (+ 1925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:01.307811 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:01.307811 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:01.307817 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:01.309187 (+ 1370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:01.309478 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:01.309478 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:01.309479 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:01.309526 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:01.309665 (+ 139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:01.309666 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:01.309666 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:01.309754 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:01.309820 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:01.309908 (+ 88us) server_negotiation.cc:299] Negotiation successful
0619 02:40:01.309961 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 02:40:01.310873 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 11723
I20260619 02:40:01.315464 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 11857
I20260619 02:40:01.319924 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 11991
I20260619 02:40:01.324167 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 11629
2026-06-19T02:40:01Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1907 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:40:01 dist-test-slave-zpbg krb5kdc[12140](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:40:01 dist-test-slave-zpbg krb5kdc[12140](info): set up 2 sockets
Jun 19 02:40:01 dist-test-slave-zpbg krb5kdc[12140](info): commencing operation
krb5kdc: starting...
W20260619 02:40:02.367031 6085 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Jun 19 02:40:02 dist-test-slave-zpbg krb5kdc[12140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836802, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:40:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:40:02Z Disabled control of system clock
WARNING: no policy specified for oryx/127.5.241.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.241.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.241.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.126.keytab.
Entry for principal oryx/127.5.241.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.126.keytab.
I20260619 02:40:02.486421 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:45231
--webserver_interface=127.5.241.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.126.keytab
--principal=oryx/127.5.241.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:44915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:45231
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:40:02.568094 12153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:02.568331 12153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:02.568369 12153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:02.569530 12153 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:40:02.569603 12153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:02.569631 12153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:02.569664 12153 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:40:02.569689 12153 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:40:02.571286 12153 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:44915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:45231
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.126.keytab
--principal=oryx/127.5.241.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:45231
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.241.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12153
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:40:02.571554 12153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:02.571789 12153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:02.576819 12158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:02.576831 12159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:02.576901 12153 server_base.cc:1034] running on GCE node
W20260619 02:40:02.576828 12161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:02.577517 12153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:02.577868 12153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:02.579072 12153 hybrid_clock.cc:648] HybridClock initialized: now 1781836802579065 us; error 39 us; skew 500 ppm
Jun 19 02:40:02 dist-test-slave-zpbg krb5kdc[12140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836802, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.241.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:02.581904 12153 init.cc:377] Logged in from keytab as oryx/127.5.241.126@KRBTEST.COM (short username oryx)
I20260619 02:40:02.582479 12153 webserver.cc:460] Webserver started at http://127.5.241.126:35909/ using document root <none> and password file <none>
I20260619 02:40:02.582737 12153 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:02.582847 12153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:02.582993 12153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:40:02.584131 12153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-0/data/instance:
uuid: "0305088fb8294ea485bdf6424e661096"
format_stamp: "Formatted at 2026-06-19 02:40:02 on dist-test-slave-zpbg"
server_key: "b77591bce0d9c7285d7cf4290bae549c"
server_key_iv: "90516066b3acc1a68d33922b6201e4a4"
server_key_version: "encryptionkey@0"
I20260619 02:40:02.584532 12153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-0/wal/instance:
uuid: "0305088fb8294ea485bdf6424e661096"
format_stamp: "Formatted at 2026-06-19 02:40:02 on dist-test-slave-zpbg"
server_key: "b77591bce0d9c7285d7cf4290bae549c"
server_key_iv: "90516066b3acc1a68d33922b6201e4a4"
server_key_version: "encryptionkey@0"
I20260619 02:40:02.586395 12153 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:40:02.587319 12168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:02.587636 12153 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 02:40:02.587754 12153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "0305088fb8294ea485bdf6424e661096"
format_stamp: "Formatted at 2026-06-19 02:40:02 on dist-test-slave-zpbg"
server_key: "b77591bce0d9c7285d7cf4290bae549c"
server_key_iv: "90516066b3acc1a68d33922b6201e4a4"
server_key_version: "encryptionkey@0"
I20260619 02:40:02.587845 12153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:02.604168 12153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:02.606531 12153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:02.606698 12153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:02.610688 12153 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.126:45231
I20260619 02:40:02.610725 12220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.126:45231 every 8 connection(s)
I20260619 02:40:02.611102 12153 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:40:02.611325 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 12153
I20260619 02:40:02.611442 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-0/wal/instance
I20260619 02:40:02.611666 6085 external_mini_cluster.cc:1428] Setting key 9d5fbb96caf3ed027756de0321847eb6
I20260619 02:40:02.612299 12221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:02.615242 12221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096: Bootstrap starting.
I20260619 02:40:02.615907 12221 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:02.616186 12221 log.cc:824] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096: Log is configured to *not* fsync() on all Append() calls
I20260619 02:40:02.616956 12221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096: No bootstrap required, opened a new log
Jun 19 02:40:02 dist-test-slave-zpbg krb5kdc[12140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836802, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM
I20260619 02:40:02.619179 12221 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } }
I20260619 02:40:02.619311 12221 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:02.619335 12221 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0305088fb8294ea485bdf6424e661096, State: Initialized, Role: FOLLOWER
I20260619 02:40:02.619496 12221 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } }
I20260619 02:40:02.619588 12221 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:40:02.619624 12221 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:40:02.619678 12221 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:02.620381 12221 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } }
I20260619 02:40:02.620483 12221 leader_election.cc:302] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0305088fb8294ea485bdf6424e661096; no voters:
I20260619 02:40:02.620630 12221 leader_election.cc:288] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:40:02.620694 12226 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:40:02.620877 12226 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096 [term 1 LEADER]: Becoming Leader. State: Replica: 0305088fb8294ea485bdf6424e661096, State: Running, Role: LEADER
I20260619 02:40:02.620896 12221 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:40:02.620970 12226 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } }
I20260619 02:40:02.621304 12226 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0305088fb8294ea485bdf6424e661096. Latest consensus state: current_term: 1 leader_uuid: "0305088fb8294ea485bdf6424e661096" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } } }
I20260619 02:40:02.621393 12226 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096 [sys.catalog]: This master's current role is: LEADER
I20260619 02:40:02.621313 12228 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0305088fb8294ea485bdf6424e661096" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } } }
I20260619 02:40:02.621490 12228 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096 [sys.catalog]: This master's current role is: LEADER
W20260619 02:40:02.622850 12242 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:40:02.622920 12242 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:40:02.622932 12224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:02.612197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50986 (local address 127.5.241.126:45231)
0619 02:40:02.612402 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:02.612410 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:02.612435 (+ 25us) server_negotiation.cc:407] Connection header received
0619 02:40:02.612861 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:02.612867 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:02.612910 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:02.613021 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:02.613832 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:02.614730 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:02.615813 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:02.616337 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:02.616466 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:02.618970 (+ 2504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:02.618974 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:02.618974 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:02.618984 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:02.620985 (+ 2001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:02.621434 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:02.621437 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:02.621438 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:02.621559 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:02.621789 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:02.621789 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:02.621790 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:02.622124 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:02.622329 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:02.622558 (+ 229us) server_negotiation.cc:299] Negotiation successful
0619 02:40:02.622700 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":84,"threads_started":1}
I20260619 02:40:02.623018 12233 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:40:02.623152 12233 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:40:02.624540 12233 catalog_manager.cc:1347] Generated new cluster ID: c32bc2d7e2e547d6adb2678ff8bbe53c
I20260619 02:40:02.624593 12233 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:40:02.648010 12233 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:40:02.648742 12233 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:40:02.652577 12233 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096: Generated new TSK 0
I20260619 02:40:02.652756 12233 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.5.241.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.241.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.241.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.65.keytab.
Entry for principal oryx/127.5.241.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.65.keytab.
I20260619 02:40:02.686062 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.65:0
--local_ip_for_outbound_sockets=127.5.241.65
--webserver_interface=127.5.241.65
--webserver_port=0
--tserver_master_addrs=127.5.241.126:45231
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.65.keytab
--principal=oryx/127.5.241.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:44915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:40:02.767011 12247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:02.767216 12247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:02.767241 12247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:02.768415 12247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:02.768458 12247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:02.768484 12247 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.65
I20260619 02:40:02.769929 12247 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:44915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.65.keytab
--principal=oryx/127.5.241.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.241.65
--webserver_port=0
--tserver_master_addrs=127.5.241.126:45231
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12247
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.65
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:40:02.770182 12247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:02.770426 12247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:02.775164 12252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:02.775207 12255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:02.775269 12253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:02.775563 12247 server_base.cc:1034] running on GCE node
I20260619 02:40:02.775774 12247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:02.776005 12247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:02.777158 12247 hybrid_clock.cc:648] HybridClock initialized: now 1781836802777155 us; error 38 us; skew 500 ppm
Jun 19 02:40:02 dist-test-slave-zpbg krb5kdc[12140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836802, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.241.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:02.779623 12247 init.cc:377] Logged in from keytab as oryx/127.5.241.65@KRBTEST.COM (short username oryx)
I20260619 02:40:02.780042 12247 webserver.cc:460] Webserver started at http://127.5.241.65:45201/ using document root <none> and password file <none>
I20260619 02:40:02.780261 12247 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:02.780347 12247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:02.780457 12247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:40:02.781445 12247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-0/data/instance:
uuid: "e90938d5560646d08f1a8872b88e0c40"
format_stamp: "Formatted at 2026-06-19 02:40:02 on dist-test-slave-zpbg"
server_key: "f1e7944be6dcdf6ecc04296a6b03e6a2"
server_key_iv: "ed188944cafe9d2661c55c074f3c1d6f"
server_key_version: "encryptionkey@0"
I20260619 02:40:02.781736 12247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-0/wal/instance:
uuid: "e90938d5560646d08f1a8872b88e0c40"
format_stamp: "Formatted at 2026-06-19 02:40:02 on dist-test-slave-zpbg"
server_key: "f1e7944be6dcdf6ecc04296a6b03e6a2"
server_key_iv: "ed188944cafe9d2661c55c074f3c1d6f"
server_key_version: "encryptionkey@0"
I20260619 02:40:02.783013 12247 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:40:02.783705 12262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:02.783905 12247 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:40:02.783991 12247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-0/wal
uuid: "e90938d5560646d08f1a8872b88e0c40"
format_stamp: "Formatted at 2026-06-19 02:40:02 on dist-test-slave-zpbg"
server_key: "f1e7944be6dcdf6ecc04296a6b03e6a2"
server_key_iv: "ed188944cafe9d2661c55c074f3c1d6f"
server_key_version: "encryptionkey@0"
I20260619 02:40:02.784062 12247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:02.801354 12247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:02.803442 12247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:02.803580 12247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:02.803812 12247 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:40:02.804111 12247 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:40:02.804143 12247 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:02.804164 12247 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:40:02.804209 12247 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:02.810035 12247 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.65:44389
I20260619 02:40:02.810081 12375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.65:44389 every 8 connection(s)
I20260619 02:40:02.810384 12247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
I20260619 02:40:02.810842 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 12247
I20260619 02:40:02.810941 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-0/wal/instance
I20260619 02:40:02.811162 6085 external_mini_cluster.cc:1428] Setting key dbcdbe61ccf6f544e62e03404129cc88
Jun 19 02:40:02 dist-test-slave-zpbg krb5kdc[12140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836802, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.241.65@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM
WARNING: no policy specified for oryx/127.5.241.66@KRBTEST.COM; defaulting to no policy
I20260619 02:40:02.820322 12224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:02.811100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:34701 (local address 127.5.241.126:45231)
0619 02:40:02.811147 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:02.811149 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:02.811441 (+ 292us) server_negotiation.cc:407] Connection header received
0619 02:40:02.812095 (+ 654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:02.812096 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:02.812108 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:02.812148 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:02.813189 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:02.813677 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:02.814743 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:02.814997 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:02.815054 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:02.817442 (+ 2388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:02.817443 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:02.817444 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:02.817451 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:02.818905 (+ 1454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:02.819322 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:02.819323 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:02.819323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:02.819383 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:02.819640 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:02.819641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:02.819641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:02.819754 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:02.819831 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:02.820138 (+ 307us) server_negotiation.cc:299] Negotiation successful
0619 02:40:02.820228 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 02:40:02.820515 12378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:02.811104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45231 (local address 127.5.241.65:34701)
0619 02:40:02.811314 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:02.811365 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:02.811991 (+ 626us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:02.812225 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:02.812229 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:02.812811 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:02.813085 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:02.813093 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:02.813813 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:02.813814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:02.814665 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:02.814667 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:02.815076 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:02.815078 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:02.815228 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:02.815753 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:02.815763 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:02.817335 (+ 1572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:02.818999 (+ 1664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:02.819005 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:02.819007 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:02.819233 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:02.819451 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:02.819452 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:02.819452 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:02.819563 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:02.819828 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:02.819835 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:02.820004 (+ 169us) client_negotiation.cc:769] Sending connection context
0619 02:40:02.820117 (+ 113us) client_negotiation.cc:240] Negotiation successful
0619 02:40:02.820269 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":77,"threads_started":1}
I20260619 02:40:02.821244 12376 heartbeater.cc:347] Connected to a master server at 127.5.241.126:45231
I20260619 02:40:02.821327 12376 heartbeater.cc:464] Registering TS with master...
I20260619 02:40:02.821550 12376 heartbeater.cc:511] Master 127.5.241.126:45231 requested a full tablet report, sending...
I20260619 02:40:02.822124 12185 ts_manager.cc:194] Registered new tserver with Master: e90938d5560646d08f1a8872b88e0c40 (127.5.241.65:44389)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.241.66@KRBTEST.COM" created.
I20260619 02:40:02.822922 12185 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.241.65@KRBTEST.COM'} at 127.5.241.65:34701
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.241.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.66.keytab.
Entry for principal oryx/127.5.241.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.66.keytab.
I20260619 02:40:02.837641 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.66:0
--local_ip_for_outbound_sockets=127.5.241.66
--webserver_interface=127.5.241.66
--webserver_port=0
--tserver_master_addrs=127.5.241.126:45231
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.66.keytab
--principal=oryx/127.5.241.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:44915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:40:02.918582 12381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:02.918838 12381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:02.918876 12381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:02.920102 12381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:02.920145 12381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:02.920172 12381 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.66
I20260619 02:40:02.921864 12381 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:44915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.66.keytab
--principal=oryx/127.5.241.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.241.66
--webserver_port=0
--tserver_master_addrs=127.5.241.126:45231
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12381
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.66
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:40:02.922120 12381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:02.922367 12381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:02.927562 12387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:02.927568 12389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:02.927574 12386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:02.927918 12381 server_base.cc:1034] running on GCE node
I20260619 02:40:02.928170 12381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:02.928437 12381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:02.929612 12381 hybrid_clock.cc:648] HybridClock initialized: now 1781836802929602 us; error 58 us; skew 500 ppm
Jun 19 02:40:02 dist-test-slave-zpbg krb5kdc[12140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836802, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.241.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:02.932354 12381 init.cc:377] Logged in from keytab as oryx/127.5.241.66@KRBTEST.COM (short username oryx)
I20260619 02:40:02.932734 12381 webserver.cc:460] Webserver started at http://127.5.241.66:44283/ using document root <none> and password file <none>
I20260619 02:40:02.932927 12381 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:02.932971 12381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:02.933058 12381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:40:02.934053 12381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-1/data/instance:
uuid: "f277ace414904d888dfcd10ba52bc2c5"
format_stamp: "Formatted at 2026-06-19 02:40:02 on dist-test-slave-zpbg"
server_key: "9c663800930cb639b2182a72b1e14e7e"
server_key_iv: "8ca9cbb3d5e947244c630e7911cabc4c"
server_key_version: "encryptionkey@0"
I20260619 02:40:02.934355 12381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-1/wal/instance:
uuid: "f277ace414904d888dfcd10ba52bc2c5"
format_stamp: "Formatted at 2026-06-19 02:40:02 on dist-test-slave-zpbg"
server_key: "9c663800930cb639b2182a72b1e14e7e"
server_key_iv: "8ca9cbb3d5e947244c630e7911cabc4c"
server_key_version: "encryptionkey@0"
I20260619 02:40:02.935734 12381 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:40:02.936465 12396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:02.936681 12381 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:40:02.936776 12381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-1/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-1/wal
uuid: "f277ace414904d888dfcd10ba52bc2c5"
format_stamp: "Formatted at 2026-06-19 02:40:02 on dist-test-slave-zpbg"
server_key: "9c663800930cb639b2182a72b1e14e7e"
server_key_iv: "8ca9cbb3d5e947244c630e7911cabc4c"
server_key_version: "encryptionkey@0"
I20260619 02:40:02.936838 12381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:02.967118 12381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:02.969360 12381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:02.969518 12381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:02.969725 12381 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:40:02.970083 12381 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:40:02.970119 12381 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:02.970144 12381 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:40:02.970202 12381 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:02.976636 12381 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.66:36493
I20260619 02:40:02.976698 12509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.66:36493 every 8 connection(s)
I20260619 02:40:02.977046 12381 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
I20260619 02:40:02.982658 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 12381
I20260619 02:40:02.982779 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-1/wal/instance
I20260619 02:40:02.983022 6085 external_mini_cluster.cc:1428] Setting key b64c122ab9269c13983200589bcb6454
Jun 19 02:40:02 dist-test-slave-zpbg krb5kdc[12140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836802, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.241.66@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM
I20260619 02:40:02.987715 12512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:02.977761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45231 (local address 127.5.241.66:41169)
0619 02:40:02.977976 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:02.978033 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:02.978574 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:02.978875 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:02.978879 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:02.979510 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:02.979713 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:02.979717 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:02.980532 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:02.980533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:02.981447 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:02.981449 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:02.981898 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:02.981899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:02.982042 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:02.982563 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:02.982572 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:02.984367 (+ 1795us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:02.986072 (+ 1705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:02.986080 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:02.986083 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:02.986429 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:02.986681 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:02.986682 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:02.986683 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:02.986849 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:02.987142 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:02.987146 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:02.987265 (+ 119us) client_negotiation.cc:769] Sending connection context
0619 02:40:02.987367 (+ 102us) client_negotiation.cc:240] Negotiation successful
0619 02:40:02.987491 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":90,"threads_started":1}
I20260619 02:40:02.987721 12224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:02.977845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:41169 (local address 127.5.241.126:45231)
0619 02:40:02.977940 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:02.977943 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:02.978123 (+ 180us) server_negotiation.cc:407] Connection header received
0619 02:40:02.978687 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:02.978688 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:02.978700 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:02.978756 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:02.979840 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:02.980424 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:02.981564 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:02.981796 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:02.981888 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:02.984464 (+ 2576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:02.984466 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:02.984466 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:02.984473 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:02.985989 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:02.986509 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:02.986510 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:02.986510 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:02.986600 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:02.986920 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:02.986920 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:02.986921 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:02.987075 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:02.987146 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:02.987443 (+ 297us) server_negotiation.cc:299] Negotiation successful
0619 02:40:02.987560 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 02:40:02.988210 12510 heartbeater.cc:347] Connected to a master server at 127.5.241.126:45231
I20260619 02:40:02.988368 12510 heartbeater.cc:464] Registering TS with master...
I20260619 02:40:02.988595 12510 heartbeater.cc:511] Master 127.5.241.126:45231 requested a full tablet report, sending...
I20260619 02:40:02.989058 12185 ts_manager.cc:194] Registered new tserver with Master: f277ace414904d888dfcd10ba52bc2c5 (127.5.241.66:36493)
I20260619 02:40:02.989455 12185 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.241.66@KRBTEST.COM'} at 127.5.241.66:41169
WARNING: no policy specified for oryx/127.5.241.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.241.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.241.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.67.keytab.
Entry for principal oryx/127.5.241.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.67.keytab.
I20260619 02:40:03.011312 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.67:0
--local_ip_for_outbound_sockets=127.5.241.67
--webserver_interface=127.5.241.67
--webserver_port=0
--tserver_master_addrs=127.5.241.126:45231
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.67.keytab
--principal=oryx/127.5.241.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:44915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:40:03.092509 12515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:03.092744 12515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:03.092785 12515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:03.093963 12515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:03.094029 12515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:03.094076 12515 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.67
I20260619 02:40:03.095644 12515 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:44915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.67.keytab
--principal=oryx/127.5.241.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.241.67
--webserver_port=0
--tserver_master_addrs=127.5.241.126:45231
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12515
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.67
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:40:03.095911 12515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:03.096138 12515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:03.101387 12523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:03.101434 12520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:03.101486 12515 server_base.cc:1034] running on GCE node
W20260619 02:40:03.101449 12521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:03.101866 12515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:03.102074 12515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:03.103236 12515 hybrid_clock.cc:648] HybridClock initialized: now 1781836803103226 us; error 33 us; skew 500 ppm
Jun 19 02:40:03 dist-test-slave-zpbg krb5kdc[12140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836803, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.241.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:03.105815 12515 init.cc:377] Logged in from keytab as oryx/127.5.241.67@KRBTEST.COM (short username oryx)
I20260619 02:40:03.106253 12515 webserver.cc:460] Webserver started at http://127.5.241.67:40553/ using document root <none> and password file <none>
I20260619 02:40:03.106487 12515 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:03.106555 12515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:03.106704 12515 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:40:03.107861 12515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-2/data/instance:
uuid: "39f073a2df8741218340f6e1a84deb6a"
format_stamp: "Formatted at 2026-06-19 02:40:03 on dist-test-slave-zpbg"
server_key: "47b1e7932ebd7465390881c21c93c195"
server_key_iv: "781fc4449303bc4a1e7d812b8940f70f"
server_key_version: "encryptionkey@0"
I20260619 02:40:03.108220 12515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-2/wal/instance:
uuid: "39f073a2df8741218340f6e1a84deb6a"
format_stamp: "Formatted at 2026-06-19 02:40:03 on dist-test-slave-zpbg"
server_key: "47b1e7932ebd7465390881c21c93c195"
server_key_iv: "781fc4449303bc4a1e7d812b8940f70f"
server_key_version: "encryptionkey@0"
I20260619 02:40:03.109529 12515 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:40:03.110296 12530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:03.110488 12515 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:40:03.110584 12515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-2/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-2/wal
uuid: "39f073a2df8741218340f6e1a84deb6a"
format_stamp: "Formatted at 2026-06-19 02:40:03 on dist-test-slave-zpbg"
server_key: "47b1e7932ebd7465390881c21c93c195"
server_key_iv: "781fc4449303bc4a1e7d812b8940f70f"
server_key_version: "encryptionkey@0"
I20260619 02:40:03.110663 12515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:03.124822 12515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:03.127231 12515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:03.127399 12515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:03.127635 12515 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:40:03.127966 12515 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:40:03.128021 12515 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:03.128064 12515 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:40:03.128093 12515 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:03.133857 12515 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.67:45923
I20260619 02:40:03.133899 12643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.67:45923 every 8 connection(s)
I20260619 02:40:03.134241 12515 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
I20260619 02:40:03.136204 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 12515
I20260619 02:40:03.136276 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-2/wal/instance
I20260619 02:40:03.136466 6085 external_mini_cluster.cc:1428] Setting key 6d9bcdb904975e4f1322abe836b9ebbf
Jun 19 02:40:03 dist-test-slave-zpbg krb5kdc[12140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836803, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.241.67@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM
I20260619 02:40:03.143172 12224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:03.134974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:60693 (local address 127.5.241.126:45231)
0619 02:40:03.135050 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:03.135053 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:03.135223 (+ 170us) server_negotiation.cc:407] Connection header received
0619 02:40:03.135678 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:03.135679 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:03.135689 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:03.135739 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:03.136781 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.137484 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.138422 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.138595 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.138643 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:03.140731 (+ 2088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:03.140733 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:03.140733 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:03.140740 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:03.141997 (+ 1257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:03.142362 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:03.142363 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:03.142363 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:03.142409 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:03.142612 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:03.142613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:03.142613 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:03.142708 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:03.142757 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:03.143051 (+ 294us) server_negotiation.cc:299] Negotiation successful
0619 02:40:03.143109 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 02:40:03.143409 12646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:03.134944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45231 (local address 127.5.241.67:60693)
0619 02:40:03.135105 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:03.135140 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:03.135602 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:03.135791 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:03.135795 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:03.136434 (+ 639us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:03.136701 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:03.136708 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.137563 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.137563 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:03.138354 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:03.138356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.138661 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.138662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:03.138756 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:03.139255 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:03.139263 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:03.140651 (+ 1388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:03.142061 (+ 1410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:03.142067 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:03.142068 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:03.142275 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:03.142466 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:03.142467 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:03.142467 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:03.142560 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:03.142761 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:03.142765 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:03.142940 (+ 175us) client_negotiation.cc:769] Sending connection context
0619 02:40:03.143026 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 02:40:03.143139 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":97,"threads_started":1}
I20260619 02:40:03.143944 12644 heartbeater.cc:347] Connected to a master server at 127.5.241.126:45231
I20260619 02:40:03.144049 12644 heartbeater.cc:464] Registering TS with master...
I20260619 02:40:03.144217 12644 heartbeater.cc:511] Master 127.5.241.126:45231 requested a full tablet report, sending...
I20260619 02:40:03.144605 12185 ts_manager.cc:194] Registered new tserver with Master: 39f073a2df8741218340f6e1a84deb6a (127.5.241.67:45923)
I20260619 02:40:03.144943 12185 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.241.67@KRBTEST.COM'} at 127.5.241.67:60693
I20260619 02:40:03.147861 6085 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.5.241.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.241.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.241.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.125.keytab.
Entry for principal oryx/127.5.241.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.125.keytab.
I20260619 02:40:03.175405 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.125:38543
--webserver_interface=127.5.241.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.125.keytab
--principal=oryx/127.5.241.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:44915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:45231,127.5.241.125:38543
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:40:03.258070 12649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:03.258288 12649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:03.258311 12649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:03.259445 12649 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:40:03.259498 12649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:03.259513 12649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:03.259526 12649 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:40:03.259538 12649 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:40:03.261106 12649 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:44915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:45231,127.5.241.125:38543
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.125.keytab
--principal=oryx/127.5.241.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.125:38543
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.241.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12649
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
W20260619 02:40:03.261250 12649 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.5.241.126:45231,127.5.241.125:38543'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260619 02:40:03.261423 12649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:03.261667 12649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:03.266474 12654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:03.266609 12657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:03.266644 12655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:03.266971 12649 server_base.cc:1034] running on GCE node
I20260619 02:40:03.267279 12649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:03.267584 12649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:03.268760 12649 hybrid_clock.cc:648] HybridClock initialized: now 1781836803268718 us; error 70 us; skew 500 ppm
Jun 19 02:40:03 dist-test-slave-zpbg krb5kdc[12140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836803, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.241.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:03.271404 12649 init.cc:377] Logged in from keytab as oryx/127.5.241.125@KRBTEST.COM (short username oryx)
I20260619 02:40:03.271839 12649 webserver.cc:460] Webserver started at http://127.5.241.125:39891/ using document root <none> and password file <none>
I20260619 02:40:03.272079 12649 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:03.272147 12649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:03.272300 12649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:40:03.273366 12649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-1/data/instance:
uuid: "56c49503edc647499d6dcab7b82fcde9"
format_stamp: "Formatted at 2026-06-19 02:40:03 on dist-test-slave-zpbg"
server_key: "0d7fdb3fc73b218fac81f654d616bc3a"
server_key_iv: "d5f1cf2607b55e763a1b3c583d6045a1"
server_key_version: "encryptionkey@0"
I20260619 02:40:03.273728 12649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-1/wal/instance:
uuid: "56c49503edc647499d6dcab7b82fcde9"
format_stamp: "Formatted at 2026-06-19 02:40:03 on dist-test-slave-zpbg"
server_key: "0d7fdb3fc73b218fac81f654d616bc3a"
server_key_iv: "d5f1cf2607b55e763a1b3c583d6045a1"
server_key_version: "encryptionkey@0"
I20260619 02:40:03.275127 12649 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:40:03.275910 12664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:03.276103 12649 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:40:03.276206 12649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-1/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-1/wal
uuid: "56c49503edc647499d6dcab7b82fcde9"
format_stamp: "Formatted at 2026-06-19 02:40:03 on dist-test-slave-zpbg"
server_key: "0d7fdb3fc73b218fac81f654d616bc3a"
server_key_iv: "d5f1cf2607b55e763a1b3c583d6045a1"
server_key_version: "encryptionkey@0"
I20260619 02:40:03.276279 12649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:03.302450 12649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:03.304728 12649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:03.304998 12649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:03.309031 12649 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.125:38543
I20260619 02:40:03.309222 12716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.125:38543 every 8 connection(s)
I20260619 02:40:03.309463 12649 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-1/data/info.pb
I20260619 02:40:03.310504 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 12649
I20260619 02:40:03.310631 12717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:03.313387 12717 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:40:03 dist-test-slave-zpbg krb5kdc[12140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836802, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.241.125@KRBTEST.COM
Jun 19 02:40:03 dist-test-slave-zpbg krb5kdc[12140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836803, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.241.125@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM
I20260619 02:40:03.322129 12720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:03.311217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37382 (local address 127.5.241.125:38543)
0619 02:40:03.311391 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:03.311400 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:03.311437 (+ 37us) server_negotiation.cc:407] Connection header received
0619 02:40:03.311825 (+ 388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:03.311831 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:03.311888 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:03.312008 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:03.312831 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.314204 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.315258 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.315659 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.315785 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:03.318341 (+ 2556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:03.318347 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:03.318347 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:03.318353 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:03.320162 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:03.320603 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:03.320606 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:03.320606 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:03.320698 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:03.320945 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:03.320946 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:03.320947 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:03.321343 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:03.321476 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:03.321743 (+ 267us) server_negotiation.cc:299] Negotiation successful
0619 02:40:03.321861 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":94,"threads_started":1}
I20260619 02:40:03.322371 12721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:03.314002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45231 (local address 127.0.0.1:51002)
0619 02:40:03.314171 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:03.314204 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:03.314287 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:03.314471 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:03.314474 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:03.314852 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:03.314930 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:03.314932 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.315620 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.315621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:03.316413 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:03.316415 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.316915 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.316916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:03.316995 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:03.317496 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:03.317506 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:03.319141 (+ 1635us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:03.321056 (+ 1915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:03.321060 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:03.321063 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:03.321296 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:03.321597 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:03.321597 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:03.321598 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:03.321687 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:03.322031 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:03.322036 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:03.322083 (+ 47us) client_negotiation.cc:769] Sending connection context
0619 02:40:03.322172 (+ 89us) client_negotiation.cc:240] Negotiation successful
0619 02:40:03.322280 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":74,"threads_started":1}
I20260619 02:40:03.322467 12224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:03.314070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51002 (local address 127.5.241.126:45231)
0619 02:40:03.314143 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:03.314145 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:03.314269 (+ 124us) server_negotiation.cc:407] Connection header received
0619 02:40:03.314347 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:03.314349 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:03.314359 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:03.314449 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:03.315011 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.315502 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.316516 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.316794 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.316906 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:03.319431 (+ 2525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:03.319432 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:03.319432 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:03.319439 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:03.320988 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:03.321419 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:03.321420 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:03.321420 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:03.321492 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:03.321763 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:03.321764 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:03.321765 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:03.321969 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:03.322034 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:03.322279 (+ 245us) server_negotiation.cc:299] Negotiation successful
0619 02:40:03.322375 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 02:40:03.323072 12717 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.241.125" port: 38543 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:40:03 dist-test-slave-zpbg krb5kdc[12140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836803, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.241.125@KRBTEST.COM for oryx/127.5.241.125@KRBTEST.COM
I20260619 02:40:03.329073 12721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:03.323375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.125:38543 (local address 127.0.0.1:37396)
0619 02:40:03.323427 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:03.323438 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:03.323480 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:03.323588 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:03.323588 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:03.323694 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:03.323741 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:03.323742 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.324361 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.324362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:03.324844 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:03.324845 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.325230 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.325231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:03.325291 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:03.325650 (+ 359us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:03.325655 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:03.326820 (+ 1165us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:03.328331 (+ 1511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:03.328331 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:03.328331 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:03.328485 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:03.328628 (+ 143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:03.328628 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:03.328628 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:03.328674 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:03.328840 (+ 166us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:03.328841 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:03.328880 (+ 39us) client_negotiation.cc:769] Sending connection context
0619 02:40:03.328928 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 02:40:03.328996 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260619 02:40:03.329124 12720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:03.323429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37396 (local address 127.5.241.125:38543)
0619 02:40:03.323461 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:03.323463 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:03.323469 (+ 6us) server_negotiation.cc:407] Connection header received
0619 02:40:03.323538 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:03.323539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:03.323546 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:03.323586 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:03.323782 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.324267 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.324939 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.325130 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.325234 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:03.326899 (+ 1665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:03.326900 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:03.326900 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:03.326906 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:03.328262 (+ 1356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:03.328538 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:03.328539 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:03.328539 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:03.328579 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:03.328712 (+ 133us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:03.328712 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:03.328712 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:03.328794 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:03.328838 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:03.328943 (+ 105us) server_negotiation.cc:299] Negotiation successful
0619 02:40:03.329038 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":18}
I20260619 02:40:03.330406 12717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9: Bootstrap starting.
I20260619 02:40:03.331274 12717 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:03.331557 12717 log.cc:824] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9: Log is configured to *not* fsync() on all Append() calls
I20260619 02:40:03.332358 12717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9: No bootstrap required, opened a new log
I20260619 02:40:03.333829 12717 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } } peers { permanent_uuid: "56c49503edc647499d6dcab7b82fcde9" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 38543 } }
I20260619 02:40:03.333958 12717 raft_consensus.cc:374] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:03.333982 12717 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56c49503edc647499d6dcab7b82fcde9, State: Initialized, Role: FOLLOWER
I20260619 02:40:03.334126 12717 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } } peers { permanent_uuid: "56c49503edc647499d6dcab7b82fcde9" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 38543 } }
I20260619 02:40:03.334398 12723 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } } peers { permanent_uuid: "56c49503edc647499d6dcab7b82fcde9" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 38543 } } }
I20260619 02:40:03.334444 12717 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:40:03.334474 12723 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [sys.catalog]: This master's current role is: FOLLOWER
W20260619 02:40:03.336072 12734 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:40:03.336143 12734 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:40:03.337780 12649 master_runner.cc:428] Detected that this master 56c49503edc647499d6dcab7b82fcde9 is joining an existing cluster
I20260619 02:40:03.337822 12649 master_runner.cc:432] Initiating AddMaster RPC to add 127.5.241.125:38543
I20260619 02:40:03.345942 12741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:03.338998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.125:38543 (local address 127.0.0.1:37400)
0619 02:40:03.339129 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:03.339138 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:03.339203 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:03.339340 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:03.339340 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:03.339661 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:03.339748 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:03.339750 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.340705 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.340706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:03.341259 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:03.341268 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.341731 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.341732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:03.341813 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:03.342356 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:03.342361 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:03.342813 (+ 452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:03.344715 (+ 1902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:03.344715 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:03.344716 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:03.344961 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:03.345180 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:03.345181 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:03.345181 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:03.345239 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:03.345544 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:03.345545 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:03.345595 (+ 50us) client_negotiation.cc:769] Sending connection context
0619 02:40:03.345672 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 02:40:03.345788 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":80,"threads_started":1}
I20260619 02:40:03.346145 12720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:03.338977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37400 (local address 127.5.241.125:38543)
0619 02:40:03.339056 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:03.339058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:03.339183 (+ 125us) server_negotiation.cc:407] Connection header received
0619 02:40:03.339272 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:03.339272 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:03.339282 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:03.339327 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:03.339827 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.340578 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.341350 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.341622 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.341694 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:03.342909 (+ 1215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:03.342910 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:03.342910 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:03.342916 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:03.344642 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:03.345053 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:03.345054 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:03.345055 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:03.345112 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:03.345321 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:03.345322 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:03.345323 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:03.345461 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:03.345538 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:03.345713 (+ 175us) server_negotiation.cc:299] Negotiation successful
0619 02:40:03.345891 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":98,"server-negotiator.queue_time_us":60}
I20260619 02:40:03.350405 12224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:03.338963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51008 (local address 127.5.241.126:45231)
0619 02:40:03.339020 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:03.339022 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:03.339157 (+ 135us) server_negotiation.cc:407] Connection header received
0619 02:40:03.339229 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:03.339230 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:03.339243 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:03.339305 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:03.339861 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.340623 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.342146 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.342416 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.342450 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:03.344083 (+ 1633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:03.344085 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:03.344086 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:03.344094 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:03.348852 (+ 4758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:03.349313 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:03.349314 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:03.349315 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:03.349379 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:03.349593 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:03.349594 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:03.349594 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:03.349741 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:03.349887 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:03.350147 (+ 260us) server_negotiation.cc:299] Negotiation successful
0619 02:40:03.350284 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 02:40:03.350798 12740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:03.338906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45231 (local address 127.0.0.1:51008)
0619 02:40:03.339071 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:03.339100 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:03.339170 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:03.339359 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:03.339360 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:03.339652 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:03.339759 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:03.339761 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.341372 (+ 1611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.341373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:03.342054 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:03.342056 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.342512 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.342513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:03.342558 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:03.343503 (+ 945us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:03.343508 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:03.343982 (+ 474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:03.348955 (+ 4973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:03.348956 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:03.348957 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:03.349193 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:03.349453 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:03.349454 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:03.349454 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:03.349522 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:03.349959 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:03.349960 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:03.350010 (+ 50us) client_negotiation.cc:769] Sending connection context
0619 02:40:03.350557 (+ 547us) client_negotiation.cc:240] Negotiation successful
0619 02:40:03.350674 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":83,"threads_started":1}
Jun 19 02:40:03 dist-test-slave-zpbg krb5kdc[12140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836802, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.241.126@KRBTEST.COM for oryx/127.5.241.125@KRBTEST.COM
I20260619 02:40:03.360994 12720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:03.353651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37404 (local address 127.5.241.125:38543)
0619 02:40:03.353709 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:03.353711 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:03.353739 (+ 28us) server_negotiation.cc:407] Connection header received
0619 02:40:03.353845 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:03.353846 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:03.353856 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:03.353911 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:03.354360 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.354903 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.355686 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.355899 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.355996 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:03.358308 (+ 2312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:03.358309 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:03.358309 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:03.358316 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:03.359776 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:03.360249 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:03.360250 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:03.360250 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:03.360307 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:03.360506 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:03.360507 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:03.360507 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:03.360612 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:03.360663 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:03.360866 (+ 203us) server_negotiation.cc:299] Negotiation successful
0619 02:40:03.360931 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 02:40:03.361084 12742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:03.353491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.125:38543 (local address 127.0.0.1:37404)
0619 02:40:03.353617 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:03.353647 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:03.353768 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:03.353915 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:03.353919 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:03.354203 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:03.354289 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:03.354292 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.354988 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.354989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:03.355587 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:03.355588 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.355997 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.355998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:03.356080 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:03.356565 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:03.356570 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:03.358204 (+ 1634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:03.359864 (+ 1660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:03.359872 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:03.359875 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:03.360163 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:03.360371 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:03.360372 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:03.360372 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:03.360444 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:03.360675 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:03.360682 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:03.360750 (+ 68us) client_negotiation.cc:769] Sending connection context
0619 02:40:03.360844 (+ 94us) client_negotiation.cc:240] Negotiation successful
0619 02:40:03.360971 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":56,"threads_started":1}
I20260619 02:40:03.361670 12185 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.5.241.125:38543: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "56c49503edc647499d6dcab7b82fcde9"
member_type: NON_VOTER
last_known_addr {
host: "127.5.241.125"
port: 38543
}
attrs {
promote: true
}
}
dest_uuid: "0305088fb8294ea485bdf6424e661096"
cas_config_opid_index: -1
I20260619 02:40:03.362007 12185 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } } peers { permanent_uuid: "56c49503edc647499d6dcab7b82fcde9" member_type: NON_VOTER last_known_addr { host: "127.5.241.125" port: 38543 } attrs { promote: true } }
I20260619 02:40:03.362493 12743 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 56c49503edc647499d6dcab7b82fcde9 (127.5.241.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } } peers { permanent_uuid: "56c49503edc647499d6dcab7b82fcde9" member_type: NON_VOTER last_known_addr { host: "127.5.241.125" port: 38543 } attrs { promote: true } } }
I20260619 02:40:03.362840 12691 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:03.363461 12743 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.5.241.125:38543
I20260619 02:40:03.363703 12649 master.cc:537] Master@127.5.241.125:38543 shutting down...
I20260619 02:40:03.363780 12691 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [term 1 FOLLOWER]: Refusing update from remote peer 0305088fb8294ea485bdf6424e661096: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260619 02:40:03.363739 12744 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0305088fb8294ea485bdf6424e661096" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } } peers { permanent_uuid: "56c49503edc647499d6dcab7b82fcde9" member_type: NON_VOTER last_known_addr { host: "127.5.241.125" port: 38543 } attrs { promote: true } } }
I20260619 02:40:03.363819 12744 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096 [sys.catalog]: This master's current role is: LEADER
I20260619 02:40:03.364179 12744 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56c49503edc647499d6dcab7b82fcde9" member_type: NON_VOTER last_known_addr { host: "127.5.241.125" port: 38543 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 02:40:03.364996 12172 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260619 02:40:03.365370 12172 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096 -> Peer 56c49503edc647499d6dcab7b82fcde9 (127.5.241.125:38543): Couldn't send request to peer 56c49503edc647499d6dcab7b82fcde9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260619 02:40:03.366302 12649 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:40:03.366389 12649 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:40:03.366600 12649 tablet_replica.cc:330] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9: stopping tablet replica
I20260619 02:40:03.379422 12649 master.cc:552] Master@127.5.241.125:38543 shutdown complete.
I20260619 02:40:03.379477 12649 master_runner.cc:305] Clearing existing system tablet
I20260619 02:40:03.379611 12649 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:03.379658 12649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:03.380283 12649 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:40:03.380805 12748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:03.380951 12649 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:03.381042 12649 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:40:03.381073 12649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-1/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-1/wal
uuid: "56c49503edc647499d6dcab7b82fcde9"
format_stamp: "Formatted at 2026-06-19 02:40:03 on dist-test-slave-zpbg"
server_key: "0d7fdb3fc73b218fac81f654d616bc3a"
server_key_iv: "d5f1cf2607b55e763a1b3c583d6045a1"
server_key_version: "encryptionkey@0"
I20260619 02:40:03.382094 12649 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:40:03.384021 12649 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260619 02:40:03.384102 12649 log.cc:1196] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260619 02:40:03.384305 12649 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9: Deleting consensus metadata
I20260619 02:40:03.384438 12649 master_runner.cc:315] Copying system tablet from 127.5.241.126:45231
I20260619 02:40:03.385094 12649 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9: tablet copy: Beginning tablet copy session from remote peer at address 127.5.241.126:45231
I20260619 02:40:03.391949 12224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:03.385692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51012 (local address 127.5.241.126:45231)
0619 02:40:03.385769 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:03.385771 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:03.385890 (+ 119us) server_negotiation.cc:407] Connection header received
0619 02:40:03.385964 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:03.385965 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:03.385975 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:03.386033 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:03.386351 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.387016 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.387775 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.387981 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.388067 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:03.389105 (+ 1038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:03.389107 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:03.389107 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:03.389115 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:03.390656 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:03.391095 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:03.391096 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:03.391097 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:03.391192 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:03.391396 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:03.391397 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:03.391398 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:03.391519 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:03.391598 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:03.391762 (+ 164us) server_negotiation.cc:299] Negotiation successful
0619 02:40:03.391847 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 02:40:03.391966 12754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:03.385625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45231 (local address 127.0.0.1:51012)
0619 02:40:03.385819 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:03.385829 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:03.385906 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:03.386039 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:03.386040 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:03.386210 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:03.386271 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:03.386272 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.387106 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.387108 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:03.387655 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:03.387656 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.388068 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.388069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:03.388138 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:03.388680 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:03.388685 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:03.388985 (+ 300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:03.390797 (+ 1812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:03.390799 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:03.390799 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:03.391009 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:03.391267 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:03.391268 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:03.391268 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:03.391334 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:03.391587 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:03.391588 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:03.391634 (+ 46us) client_negotiation.cc:769] Sending connection context
0619 02:40:03.391733 (+ 99us) client_negotiation.cc:240] Negotiation successful
0619 02:40:03.391855 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":100,"threads_started":1}
I20260619 02:40:03.392338 12205 tablet_copy_service.cc:140] P 0305088fb8294ea485bdf6424e661096: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 56c49503edc647499d6dcab7b82fcde9 ({username='oryx', principal='oryx/127.5.241.125@KRBTEST.COM'} at 127.0.0.1:51012)
I20260619 02:40:03.392454 12205 tablet_copy_service.cc:161] P 0305088fb8294ea485bdf6424e661096: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 56c49503edc647499d6dcab7b82fcde9 at {username='oryx', principal='oryx/127.5.241.125@KRBTEST.COM'} at 127.0.0.1:51012: session id = 56c49503edc647499d6dcab7b82fcde9-00000000000000000000000000000000
I20260619 02:40:03.393281 12205 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096: Tablet Copy: opened 0 blocks and 1 log segments
I20260619 02:40:03.393975 12649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260619 02:40:03.395372 12649 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9: tablet copy: Starting download of 0 data blocks...
I20260619 02:40:03.395525 12649 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9: tablet copy: Starting download of 1 WAL segments...
I20260619 02:40:03.396476 12649 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260619 02:40:03.397452 12205 tablet_copy_service.cc:342] P 0305088fb8294ea485bdf6424e661096: Request end of tablet copy session 56c49503edc647499d6dcab7b82fcde9-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.5.241.125@KRBTEST.COM'} at 127.0.0.1:51012
I20260619 02:40:03.397513 12205 tablet_copy_service.cc:433] P 0305088fb8294ea485bdf6424e661096: ending tablet copy session 56c49503edc647499d6dcab7b82fcde9-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 56c49503edc647499d6dcab7b82fcde9
I20260619 02:40:03.400292 12649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:03.401904 12758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:03.401923 12755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:03.402208 12756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:03.402561 12649 server_base.cc:1034] running on GCE node
I20260619 02:40:03.402657 12649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:03.402827 12649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:03.403982 12649 hybrid_clock.cc:648] HybridClock initialized: now 1781836803403971 us; error 32 us; skew 500 ppm
Jun 19 02:40:03 dist-test-slave-zpbg krb5kdc[12140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836803, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.241.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:03.405679 12649 init.cc:377] Logged in from keytab as oryx/127.5.241.125@KRBTEST.COM (short username oryx)
I20260619 02:40:03.406071 12649 webserver.cc:460] Webserver started at http://127.5.241.125:33339/ using document root <none> and password file <none>
I20260619 02:40:03.406527 12649 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:03.406571 12649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:03.407850 12649 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:40:03.408599 12765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:03.408804 12649 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:40:03.408875 12649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-1/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-1/wal
uuid: "56c49503edc647499d6dcab7b82fcde9"
format_stamp: "Formatted at 2026-06-19 02:40:03 on dist-test-slave-zpbg"
server_key: "0d7fdb3fc73b218fac81f654d616bc3a"
server_key_iv: "d5f1cf2607b55e763a1b3c583d6045a1"
server_key_version: "encryptionkey@0"
I20260619 02:40:03.409046 12649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:03.415441 12649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:03.417337 12649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:03.420552 12649 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.125:38543
I20260619 02:40:03.420950 12817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.125:38543 every 8 connection(s)
I20260619 02:40:03.421015 12649 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-1/data/info.pb
I20260619 02:40:03.421937 12818 sys_catalog.cc:263] Verifying existing consensus state
I20260619 02:40:03.422497 12818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9: Bootstrap starting.
I20260619 02:40:03.425669 12818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:40:03.425913 12818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9: Bootstrap complete.
I20260619 02:40:03.426117 12818 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } } peers { permanent_uuid: "56c49503edc647499d6dcab7b82fcde9" member_type: NON_VOTER last_known_addr { host: "127.5.241.125" port: 38543 } attrs { promote: true } }
I20260619 02:40:03.426209 12818 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 56c49503edc647499d6dcab7b82fcde9, State: Initialized, Role: LEARNER
I20260619 02:40:03.426249 12818 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } } peers { permanent_uuid: "56c49503edc647499d6dcab7b82fcde9" member_type: NON_VOTER last_known_addr { host: "127.5.241.125" port: 38543 } attrs { promote: true } }
I20260619 02:40:03.426445 12818 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:40:03.426425 12822 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } } peers { permanent_uuid: "56c49503edc647499d6dcab7b82fcde9" member_type: NON_VOTER last_known_addr { host: "127.5.241.125" port: 38543 } attrs { promote: true } } }
I20260619 02:40:03.426638 12822 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [sys.catalog]: This master's current role is: LEARNER
I20260619 02:40:03.429185 12833 catalog_manager.cc:1259] Loaded cluster ID: c32bc2d7e2e547d6adb2678ff8bbe53c
I20260619 02:40:03.429239 12833 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9: loading cluster ID for follower catalog manager: success
I20260619 02:40:03.430460 12833 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9: acquiring CA information for follower catalog manager: success
I20260619 02:40:03.430859 12833 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 02:40:03.431803 12821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:03.424196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37436 (local address 127.5.241.125:38543)
0619 02:40:03.424331 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:03.424333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:03.424368 (+ 35us) server_negotiation.cc:407] Connection header received
0619 02:40:03.424398 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:03.424399 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:03.424407 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:03.424597 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:03.425205 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.425979 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.427052 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.427319 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.427401 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:03.428699 (+ 1298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:03.428700 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:03.428701 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:03.428708 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:03.430549 (+ 1841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:03.430984 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:03.430985 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:03.430986 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:03.431043 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:03.431267 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:03.431268 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:03.431269 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:03.431382 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:03.431491 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:03.431597 (+ 106us) server_negotiation.cc:299] Negotiation successful
0619 02:40:03.431686 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":93,"threads_started":1}
WARNING: no policy specified for oryx/127.5.241.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.241.124@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.241.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.124.keytab.
Entry for principal oryx/127.5.241.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.124.keytab.
I20260619 02:40:03.462062 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.124:36925
--webserver_interface=127.5.241.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.124.keytab
--principal=oryx/127.5.241.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:44915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:45231,127.5.241.125:38543,127.5.241.124:36925
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:40:03.546068 12836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:03.546307 12836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:03.546344 12836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:03.547605 12836 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:40:03.547673 12836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:03.547698 12836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:03.547729 12836 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:40:03.547755 12836 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:40:03.549255 12836 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:44915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:45231,127.5.241.125:38543,127.5.241.124:36925
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.124.keytab
--principal=oryx/127.5.241.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.124:36925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.241.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12836
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:40:03.549546 12836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:03.549777 12836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:03.554679 12842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:03.554714 12841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:03.554721 12844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:03.554955 12836 server_base.cc:1034] running on GCE node
I20260619 02:40:03.555223 12836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:03.555552 12836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:03.556763 12836 hybrid_clock.cc:648] HybridClock initialized: now 1781836803556778 us; error 60 us; skew 500 ppm
Jun 19 02:40:03 dist-test-slave-zpbg krb5kdc[12140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836803, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.241.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:03.559962 12836 init.cc:377] Logged in from keytab as oryx/127.5.241.124@KRBTEST.COM (short username oryx)
I20260619 02:40:03.560443 12836 webserver.cc:460] Webserver started at http://127.5.241.124:35797/ using document root <none> and password file <none>
I20260619 02:40:03.560684 12836 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:03.560751 12836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:03.560894 12836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:40:03.561947 12836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-2/data/instance:
uuid: "1a6c09a7da984a169cf84a7187990d2b"
format_stamp: "Formatted at 2026-06-19 02:40:03 on dist-test-slave-zpbg"
server_key: "15ca7cefb56df29ce6dad1570318830b"
server_key_iv: "bb6a7526a460b92dbe62301766a459b4"
server_key_version: "encryptionkey@0"
I20260619 02:40:03.562306 12836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-2/wal/instance:
uuid: "1a6c09a7da984a169cf84a7187990d2b"
format_stamp: "Formatted at 2026-06-19 02:40:03 on dist-test-slave-zpbg"
server_key: "15ca7cefb56df29ce6dad1570318830b"
server_key_iv: "bb6a7526a460b92dbe62301766a459b4"
server_key_version: "encryptionkey@0"
I20260619 02:40:03.563717 12836 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:40:03.564890 12851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:03.565074 12836 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:40:03.565171 12836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-2/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-2/wal
uuid: "1a6c09a7da984a169cf84a7187990d2b"
format_stamp: "Formatted at 2026-06-19 02:40:03 on dist-test-slave-zpbg"
server_key: "15ca7cefb56df29ce6dad1570318830b"
server_key_iv: "bb6a7526a460b92dbe62301766a459b4"
server_key_version: "encryptionkey@0"
I20260619 02:40:03.565253 12836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:03.587054 12836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:03.589388 12836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:03.589498 12836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:03.593674 12836 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.124:36925
I20260619 02:40:03.593725 12903 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.124:36925 every 8 connection(s)
I20260619 02:40:03.594117 12836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-2/data/info.pb
I20260619 02:40:03.595546 12904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:03.597100 12904 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:40:03.598064 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 12836
Jun 19 02:40:03 dist-test-slave-zpbg krb5kdc[12140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836803, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.241.124@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM
Jun 19 02:40:03 dist-test-slave-zpbg krb5kdc[12140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836802, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.241.124@KRBTEST.COM
I20260619 02:40:03.607366 12224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:03.597949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51026 (local address 127.5.241.126:45231)
0619 02:40:03.598038 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:03.598041 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:03.598154 (+ 113us) server_negotiation.cc:407] Connection header received
0619 02:40:03.599368 (+ 1214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:03.599369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:03.599382 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:03.599460 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:03.600009 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.600649 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.601568 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.601758 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.601847 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:03.604164 (+ 2317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:03.604166 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:03.604166 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:03.604173 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:03.605916 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:03.606320 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:03.606321 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:03.606322 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:03.606385 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:03.606652 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:03.606653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:03.606653 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:03.606757 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:03.606889 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:03.607186 (+ 297us) server_negotiation.cc:299] Negotiation successful
0619 02:40:03.607265 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 02:40:03.607525 12906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:03.597778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45231 (local address 127.0.0.1:51026)
0619 02:40:03.598012 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:03.598071 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:03.599269 (+ 1198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:03.599479 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:03.599482 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:03.599750 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:03.599914 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:03.599918 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.600780 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.600781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:03.601465 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:03.601467 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.601896 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.601897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:03.601959 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:03.602544 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:03.602553 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:03.604053 (+ 1500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:03.606003 (+ 1950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:03.606008 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:03.606010 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:03.606249 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:03.606455 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:03.606455 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:03.606455 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:03.606582 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:03.606883 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:03.606889 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:03.607042 (+ 153us) client_negotiation.cc:769] Sending connection context
0619 02:40:03.607154 (+ 112us) client_negotiation.cc:240] Negotiation successful
0619 02:40:03.607282 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":101,"threads_started":1}
I20260619 02:40:03.607774 12907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:03.599009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60340 (local address 127.5.241.124:36925)
0619 02:40:03.599134 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:03.599138 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:03.599174 (+ 36us) server_negotiation.cc:407] Connection header received
0619 02:40:03.599266 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:03.599272 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:03.599297 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:03.599432 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:03.600040 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.600847 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.601539 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.601841 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.601953 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:03.604694 (+ 2741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:03.604700 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:03.604701 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:03.604706 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:03.606497 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:03.606892 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:03.606895 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:03.606895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:03.606956 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:03.607162 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:03.607162 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:03.607163 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:03.607323 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:03.607520 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:03.607570 (+ 50us) server_negotiation.cc:299] Negotiation successful
0619 02:40:03.607658 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":73,"threads_started":1}
I20260619 02:40:03.608373 12904 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.241.125" port: 38543 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:40:03 dist-test-slave-zpbg krb5kdc[12140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836803, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.241.124@KRBTEST.COM for oryx/127.5.241.125@KRBTEST.COM
I20260619 02:40:03.615360 12906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:03.608640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.125:38543 (local address 127.0.0.1:37440)
0619 02:40:03.608672 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:03.608682 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:03.608714 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:03.608870 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:03.608870 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:03.609014 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:03.609044 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:03.609045 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.609784 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.609785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:03.610427 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:03.610429 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.610937 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.610939 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:03.611013 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:03.611466 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:03.611471 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:03.612828 (+ 1357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:03.614376 (+ 1548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:03.614376 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:03.614377 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:03.614570 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:03.614763 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:03.614799 (+ 36us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:03.614800 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:03.614861 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:03.615126 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:03.615127 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:03.615165 (+ 38us) client_negotiation.cc:769] Sending connection context
0619 02:40:03.615222 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 02:40:03.615296 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260619 02:40:03.615375 12821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:03.608703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37440 (local address 127.5.241.125:38543)
0619 02:40:03.608767 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:03.608769 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:03.608778 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:40:03.608801 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:03.608802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:03.608812 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:03.608862 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:03.609123 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.609704 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.610516 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.610856 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.610926 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:03.612916 (+ 1990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:03.612917 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:03.612917 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:03.612947 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:03.614307 (+ 1360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:03.614637 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:03.614638 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:03.614638 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:03.614693 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:03.614940 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:03.614941 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:03.614941 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:03.615057 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:03.615118 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:03.615237 (+ 119us) server_negotiation.cc:299] Negotiation successful
0619 02:40:03.615309 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 02:40:03.615840 12904 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.241.124" port: 36925 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:40:03 dist-test-slave-zpbg krb5kdc[12140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836803, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.241.124@KRBTEST.COM for oryx/127.5.241.124@KRBTEST.COM
I20260619 02:40:03.622188 12906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:03.616077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.124:36925 (local address 127.0.0.1:60352)
0619 02:40:03.616126 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:03.616137 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:03.616181 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:03.616364 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:03.616365 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:03.616487 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:03.616513 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:03.616515 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.617154 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.617155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:03.617614 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:03.617615 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.618005 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.618006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:03.618051 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:03.618405 (+ 354us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:03.618409 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:03.619527 (+ 1118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:03.621057 (+ 1530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:03.621058 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:03.621059 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:03.621211 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:03.621475 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:03.621475 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:03.621476 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:03.621527 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:03.621935 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:03.621936 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:03.621976 (+ 40us) client_negotiation.cc:769] Sending connection context
0619 02:40:03.622030 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 02:40:03.622101 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 02:40:03.622272 12907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:03.616126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60352 (local address 127.5.241.124:36925)
0619 02:40:03.616157 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:03.616159 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:03.616168 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:40:03.616267 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:03.616268 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:03.616278 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:03.616360 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:03.616595 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.617058 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.617705 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.617925 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.618007 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:03.619600 (+ 1593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:03.619601 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:03.619602 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:03.619607 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:03.620990 (+ 1383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:03.621294 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:03.621295 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:03.621295 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:03.621397 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:03.621605 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:03.621606 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:03.621606 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:03.621718 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:03.621791 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:03.622057 (+ 266us) server_negotiation.cc:299] Negotiation successful
0619 02:40:03.622146 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":16}
I20260619 02:40:03.623726 12904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b: Bootstrap starting.
I20260619 02:40:03.624545 12904 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:03.624825 12904 log.cc:824] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b: Log is configured to *not* fsync() on all Append() calls
I20260619 02:40:03.627074 12904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b: No bootstrap required, opened a new log
I20260619 02:40:03.628762 12904 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } } peers { permanent_uuid: "56c49503edc647499d6dcab7b82fcde9" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 38543 } } peers { permanent_uuid: "1a6c09a7da984a169cf84a7187990d2b" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 36925 } }
I20260619 02:40:03.628925 12904 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:03.628973 12904 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a6c09a7da984a169cf84a7187990d2b, State: Initialized, Role: FOLLOWER
I20260619 02:40:03.629104 12904 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } } peers { permanent_uuid: "56c49503edc647499d6dcab7b82fcde9" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 38543 } } peers { permanent_uuid: "1a6c09a7da984a169cf84a7187990d2b" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 36925 } }
I20260619 02:40:03.629449 12904 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:40:03.629467 12909 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } } peers { permanent_uuid: "56c49503edc647499d6dcab7b82fcde9" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 38543 } } peers { permanent_uuid: "1a6c09a7da984a169cf84a7187990d2b" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 36925 } } }
I20260619 02:40:03.629609 12909 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [sys.catalog]: This master's current role is: FOLLOWER
W20260619 02:40:03.631404 12920 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:40:03.631458 12920 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:40:03.634516 12836 master_runner.cc:428] Detected that this master 1a6c09a7da984a169cf84a7187990d2b is joining an existing cluster
I20260619 02:40:03.634614 12836 master_runner.cc:432] Initiating AddMaster RPC to add 127.5.241.124:36925
I20260619 02:40:03.643332 12821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:03.636128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37454 (local address 127.5.241.125:38543)
0619 02:40:03.636164 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:03.636166 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:03.636405 (+ 239us) server_negotiation.cc:407] Connection header received
0619 02:40:03.636493 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:03.636494 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:03.636504 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:03.636534 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:03.636935 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.637644 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.639275 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.639571 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.639610 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:03.640778 (+ 1168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:03.640779 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:03.640779 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:03.640786 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:03.642118 (+ 1332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:03.642464 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:03.642465 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:03.642466 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:03.642522 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:03.642792 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:03.642793 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:03.642794 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:03.642918 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:03.642998 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:03.643165 (+ 167us) server_negotiation.cc:299] Negotiation successful
0619 02:40:03.643243 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 02:40:03.643357 12927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:03.636074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.125:38543 (local address 127.0.0.1:37454)
0619 02:40:03.636355 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:03.636363 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:03.636426 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:03.636593 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:03.636594 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:03.636801 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:03.636853 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:03.636855 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.637759 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.637759 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:03.639140 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:03.639142 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.639678 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.639678 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:03.639741 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:03.640229 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:03.640234 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:03.640673 (+ 439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:03.642213 (+ 1540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:03.642214 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:03.642215 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:03.642391 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:03.642599 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:03.642600 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:03.642601 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:03.642679 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:03.642988 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:03.642989 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:03.643046 (+ 57us) client_negotiation.cc:769] Sending connection context
0619 02:40:03.643128 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 02:40:03.643236 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"mutex_wait_us":76,"thread_start_us":63,"threads_started":1}
I20260619 02:40:03.659600 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 12247
I20260619 02:40:03.664665 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.65:44389
--local_ip_for_outbound_sockets=127.5.241.65
--tserver_master_addrs=127.5.241.126:45231
--webserver_port=45201
--webserver_interface=127.5.241.65
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.65.keytab
--principal=oryx/127.5.241.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:44915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.5.241.126:45231,127.5.241.125:38543
--tserver_master_addrs=127.5.241.126:45231,127.5.241.125:38543,127.5.241.124:36925 with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:40:03.671185 12928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:03.635777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.124:36925 (local address 127.0.0.1:60364)
0619 02:40:03.636182 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:03.636194 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:03.636254 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:03.636426 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:03.636427 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:03.636717 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:03.636818 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:03.636820 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.637726 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.637727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:03.639122 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:03.639125 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.639625 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.639625 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:03.639684 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:03.667305 (+ 27621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:03.667313 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:03.667869 (+ 556us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:03.669851 (+ 1982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:03.669852 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:03.669852 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:03.670079 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:03.670277 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:03.670278 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:03.670278 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:03.670337 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:03.670684 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:03.670684 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:03.670738 (+ 54us) client_negotiation.cc:769] Sending connection context
0619 02:40:03.670841 (+ 103us) client_negotiation.cc:240] Negotiation successful
0619 02:40:03.671013 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"mutex_wait_us":10,"thread_start_us":87,"threads_started":1}
I20260619 02:40:03.671784 12907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:03.635867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60364 (local address 127.5.241.124:36925)
0619 02:40:03.635899 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:03.635901 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:03.636308 (+ 407us) server_negotiation.cc:407] Connection header received
0619 02:40:03.636328 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:03.636329 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:03.636339 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:03.636386 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:03.636899 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.637644 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.639207 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.639507 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.639564 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:03.667979 (+ 28415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:03.667981 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:03.667981 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:03.667987 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:03.669751 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:03.670160 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:03.670161 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:03.670162 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:03.670211 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:03.670440 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:03.670440 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:03.670441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:03.670594 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:03.670660 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:03.671560 (+ 900us) server_negotiation.cc:299] Negotiation successful
0619 02:40:03.671646 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 02:40:03.671994 12224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:03.635759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51036 (local address 127.5.241.126:45231)
0619 02:40:03.635837 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:03.635839 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:03.636246 (+ 407us) server_negotiation.cc:407] Connection header received
0619 02:40:03.636328 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:03.636329 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:03.636340 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:03.636400 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:03.636911 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.637669 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.639336 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.639630 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.639673 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:03.668530 (+ 28857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:03.668532 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:03.668532 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:03.668542 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:03.670596 (+ 2054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:03.671123 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:03.671125 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:03.671125 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:03.671185 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:03.671425 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:03.671426 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:03.671426 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:03.671558 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:03.671609 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:03.671815 (+ 206us) server_negotiation.cc:299] Negotiation successful
0619 02:40:03.671898 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 02:40:03.672313 12926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:03.636016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45231 (local address 127.0.0.1:51036)
0619 02:40:03.636182 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:03.636194 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:03.636254 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:03.636448 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:03.636449 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:03.636729 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:03.636813 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:03.636816 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.637765 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.637766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:03.639231 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:03.639233 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.639752 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.639753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:03.639815 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:03.667573 (+ 27758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:03.667579 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:03.668164 (+ 585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:03.670702 (+ 2538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:03.670703 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:03.670703 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:03.671028 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:03.671287 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:03.671288 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:03.671288 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:03.671349 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:03.671638 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:03.671639 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:03.671692 (+ 53us) client_negotiation.cc:769] Sending connection context
0619 02:40:03.672104 (+ 412us) client_negotiation.cc:240] Negotiation successful
0619 02:40:03.672208 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":67,"thread_start_us":67,"threads_started":1}
I20260619 02:40:03.675235 12909 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:40:03.675324 12909 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } } peers { permanent_uuid: "56c49503edc647499d6dcab7b82fcde9" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 38543 } } peers { permanent_uuid: "1a6c09a7da984a169cf84a7187990d2b" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 36925 } }
I20260619 02:40:03.675582 12909 leader_election.cc:288] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0305088fb8294ea485bdf6424e661096 (127.5.241.126:45231), 56c49503edc647499d6dcab7b82fcde9 (127.5.241.125:38543)
I20260619 02:40:03.675714 12195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a6c09a7da984a169cf84a7187990d2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0305088fb8294ea485bdf6424e661096" is_pre_election: true
I20260619 02:40:03.675952 12792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a6c09a7da984a169cf84a7187990d2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56c49503edc647499d6dcab7b82fcde9" is_pre_election: true
I20260619 02:40:03.676069 12792 raft_consensus.cc:1799] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [term 1 LEARNER]: Handling vote request from an unknown peer 1a6c09a7da984a169cf84a7187990d2b
I20260619 02:40:03.676124 12792 raft_consensus.cc:2396] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [term 1 LEARNER]: Leader pre-election vote request: Denying vote to candidate 1a6c09a7da984a169cf84a7187990d2b for term 1 because replica has last-logged OpId of term: 1 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260619 02:40:03.676435 12855 leader_election.cc:302] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1a6c09a7da984a169cf84a7187990d2b; no voters: 0305088fb8294ea485bdf6424e661096, 56c49503edc647499d6dcab7b82fcde9
I20260619 02:40:03.676673 12909 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:03.677585 12909 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
Jun 19 02:40:03 dist-test-slave-zpbg krb5kdc[12140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836802, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.241.126@KRBTEST.COM for oryx/127.5.241.124@KRBTEST.COM
I20260619 02:40:03.682730 12742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:03.674404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.124:36925 (local address 127.0.0.1:60380)
0619 02:40:03.674462 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:03.674474 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:03.674516 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:03.674904 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:03.674905 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:03.675066 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:03.675112 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:03.675113 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.676524 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.676525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:03.677350 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:03.677353 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.677790 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.677791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:03.677871 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:03.678280 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:03.678285 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:03.679843 (+ 1558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:03.681696 (+ 1853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:03.681697 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:03.681698 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:03.681917 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:03.682115 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:03.682116 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:03.682116 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:03.682168 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:03.682427 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:03.682427 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:03.682475 (+ 48us) client_negotiation.cc:769] Sending connection context
0619 02:40:03.682524 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 02:40:03.682629 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260619 02:40:03.683139 12907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:03.674593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60380 (local address 127.5.241.124:36925)
0619 02:40:03.674657 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:03.674659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:03.674667 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:40:03.674701 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:03.674710 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:03.674722 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:03.674899 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:03.675217 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.676438 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.677444 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.677728 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.677790 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:03.679953 (+ 2163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:03.679954 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:03.679954 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:03.679962 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:03.681596 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:03.681996 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:03.681997 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:03.681998 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:03.682048 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:03.682232 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:03.682232 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:03.682233 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:03.682355 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:03.682943 (+ 588us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:03.682984 (+ 41us) server_negotiation.cc:299] Negotiation successful
0619 02:40:03.683054 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 02:40:03.683570 12185 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.5.241.124:36925: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "1a6c09a7da984a169cf84a7187990d2b"
member_type: NON_VOTER
last_known_addr {
host: "127.5.241.124"
port: 36925
}
attrs {
promote: true
}
}
dest_uuid: "0305088fb8294ea485bdf6424e661096"
cas_config_opid_index: 5
I20260619 02:40:03.683820 12185 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } } peers { permanent_uuid: "56c49503edc647499d6dcab7b82fcde9" member_type: NON_VOTER last_known_addr { host: "127.5.241.125" port: 38543 } attrs { promote: true } } peers { permanent_uuid: "1a6c09a7da984a169cf84a7187990d2b" member_type: NON_VOTER last_known_addr { host: "127.5.241.124" port: 36925 } attrs { promote: true } }
I20260619 02:40:03.684154 12744 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 1a6c09a7da984a169cf84a7187990d2b (127.5.241.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } } peers { permanent_uuid: "56c49503edc647499d6dcab7b82fcde9" member_type: NON_VOTER last_known_addr { host: "127.5.241.125" port: 38543 } attrs { promote: true } } peers { permanent_uuid: "1a6c09a7da984a169cf84a7187990d2b" member_type: NON_VOTER last_known_addr { host: "127.5.241.124" port: 36925 } attrs { promote: true } } }
I20260619 02:40:03.685127 12744 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.5.241.124:36925
I20260619 02:40:03.685204 12744 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0305088fb8294ea485bdf6424e661096" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } } peers { permanent_uuid: "56c49503edc647499d6dcab7b82fcde9" member_type: NON_VOTER last_known_addr { host: "127.5.241.125" port: 38543 } attrs { promote: true } } peers { permanent_uuid: "1a6c09a7da984a169cf84a7187990d2b" member_type: NON_VOTER last_known_addr { host: "127.5.241.124" port: 36925 } attrs { promote: true } } }
I20260619 02:40:03.685266 12744 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096 [sys.catalog]: This master's current role is: LEADER
I20260619 02:40:03.685374 12836 master.cc:537] Master@127.5.241.124:36925 shutting down...
W20260619 02:40:03.686230 12169 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096 -> Peer 1a6c09a7da984a169cf84a7187990d2b (127.5.241.124:36925): Couldn't send request to peer 1a6c09a7da984a169cf84a7187990d2b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260619 02:40:03.690714 12742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:03.685452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.125:38543 (local address 127.0.0.1:37462)
0619 02:40:03.686365 (+ 913us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:03.686377 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:03.686432 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:03.686613 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:03.686614 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:03.686616 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:40:03.686660 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:03.686662 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.687926 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.687927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:03.689186 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:03.689189 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.690450 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.690451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:03.690534 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:03.690556 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 02:40:03.690608 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 02:40:03.690643 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":896}
I20260619 02:40:03.690996 12821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:03.685633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37462 (local address 127.5.241.125:38543)
0619 02:40:03.685746 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:03.685748 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:03.686440 (+ 692us) server_negotiation.cc:407] Connection header received
0619 02:40:03.686490 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:03.686491 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:03.686515 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:03.686578 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:40:03.686747 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.687558 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.689441 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.690047 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.690115 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:03.690203 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:03.690889 (+ 686us) server_negotiation.cc:299] Negotiation successful
0619 02:40:03.690932 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260619 02:40:03.691200 12792 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [term 1 LEARNER]: Refusing update from remote peer 0305088fb8294ea485bdf6424e661096: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260619 02:40:03.691409 12743 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56c49503edc647499d6dcab7b82fcde9" member_type: NON_VOTER last_known_addr { host: "127.5.241.125" port: 38543 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260619 02:40:03.691923 12792 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 1a6c09a7da984a169cf84a7187990d2b (127.5.241.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } } peers { permanent_uuid: "56c49503edc647499d6dcab7b82fcde9" member_type: NON_VOTER last_known_addr { host: "127.5.241.125" port: 38543 } attrs { promote: true } } peers { permanent_uuid: "1a6c09a7da984a169cf84a7187990d2b" member_type: NON_VOTER last_known_addr { host: "127.5.241.124" port: 36925 } attrs { promote: true } } }
I20260619 02:40:03.693212 12822 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0305088fb8294ea485bdf6424e661096. Latest consensus state: current_term: 1 leader_uuid: "0305088fb8294ea485bdf6424e661096" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } } peers { permanent_uuid: "56c49503edc647499d6dcab7b82fcde9" member_type: NON_VOTER last_known_addr { host: "127.5.241.125" port: 38543 } attrs { promote: true } } peers { permanent_uuid: "1a6c09a7da984a169cf84a7187990d2b" member_type: NON_VOTER last_known_addr { host: "127.5.241.124" port: 36925 } attrs { promote: true } } }
I20260619 02:40:03.693301 12822 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [sys.catalog]: This master's current role is: LEARNER
I20260619 02:40:03.693339 12822 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0305088fb8294ea485bdf6424e661096" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } } peers { permanent_uuid: "56c49503edc647499d6dcab7b82fcde9" member_type: NON_VOTER last_known_addr { host: "127.5.241.125" port: 38543 } attrs { promote: true } } peers { permanent_uuid: "1a6c09a7da984a169cf84a7187990d2b" member_type: NON_VOTER last_known_addr { host: "127.5.241.124" port: 36925 } attrs { promote: true } } }
I20260619 02:40:03.693387 12822 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [sys.catalog]: This master's current role is: LEARNER
I20260619 02:40:03.693598 12743 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096: attempting to promote NON_VOTER 56c49503edc647499d6dcab7b82fcde9 to VOTER
I20260619 02:40:03.693770 12743 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } } peers { permanent_uuid: "56c49503edc647499d6dcab7b82fcde9" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 38543 } attrs { promote: false } } peers { permanent_uuid: "1a6c09a7da984a169cf84a7187990d2b" member_type: NON_VOTER last_known_addr { host: "127.5.241.124" port: 36925 } attrs { promote: true } }
I20260619 02:40:03.694295 12792 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [term 1 LEARNER]: Refusing update from remote peer 0305088fb8294ea485bdf6424e661096: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260619 02:40:03.694451 12744 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56c49503edc647499d6dcab7b82fcde9" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 38543 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260619 02:40:03.694523 12169 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096 -> Peer 1a6c09a7da984a169cf84a7187990d2b (127.5.241.124:36925): Couldn't send request to peer 1a6c09a7da984a169cf84a7187990d2b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260619 02:40:03.695048 12743 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 56c49503edc647499d6dcab7b82fcde9 (127.5.241.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } } peers { permanent_uuid: "56c49503edc647499d6dcab7b82fcde9" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 38543 } attrs { promote: false } } peers { permanent_uuid: "1a6c09a7da984a169cf84a7187990d2b" member_type: NON_VOTER last_known_addr { host: "127.5.241.124" port: 36925 } attrs { promote: true } } }
I20260619 02:40:03.695166 12792 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 56c49503edc647499d6dcab7b82fcde9 (127.5.241.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } } peers { permanent_uuid: "56c49503edc647499d6dcab7b82fcde9" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 38543 } attrs { promote: false } } peers { permanent_uuid: "1a6c09a7da984a169cf84a7187990d2b" member_type: NON_VOTER last_known_addr { host: "127.5.241.124" port: 36925 } attrs { promote: true } } }
I20260619 02:40:03.695900 12743 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0305088fb8294ea485bdf6424e661096" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } } peers { permanent_uuid: "56c49503edc647499d6dcab7b82fcde9" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 38543 } attrs { promote: false } } peers { permanent_uuid: "1a6c09a7da984a169cf84a7187990d2b" member_type: NON_VOTER last_known_addr { host: "127.5.241.124" port: 36925 } attrs { promote: true } } }
I20260619 02:40:03.695972 12743 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096 [sys.catalog]: This master's current role is: LEADER
I20260619 02:40:03.696019 12743 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0305088fb8294ea485bdf6424e661096" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } } peers { permanent_uuid: "56c49503edc647499d6dcab7b82fcde9" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 38543 } attrs { promote: false } } peers { permanent_uuid: "1a6c09a7da984a169cf84a7187990d2b" member_type: NON_VOTER last_known_addr { host: "127.5.241.124" port: 36925 } attrs { promote: true } } }
I20260619 02:40:03.696066 12743 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096 [sys.catalog]: This master's current role is: LEADER
I20260619 02:40:03.696348 12931 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0305088fb8294ea485bdf6424e661096" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } } peers { permanent_uuid: "56c49503edc647499d6dcab7b82fcde9" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 38543 } attrs { promote: false } } peers { permanent_uuid: "1a6c09a7da984a169cf84a7187990d2b" member_type: NON_VOTER last_known_addr { host: "127.5.241.124" port: 36925 } attrs { promote: true } } }
I20260619 02:40:03.696590 12931 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:40:03.696619 12836 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:40:03.696694 12836 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:40:03.696728 12836 tablet_replica.cc:330] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b: stopping tablet replica
I20260619 02:40:03.699438 12836 master.cc:552] Master@127.5.241.124:36925 shutdown complete.
I20260619 02:40:03.699486 12836 master_runner.cc:305] Clearing existing system tablet
I20260619 02:40:03.699607 12836 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:03.699651 12836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:03.700364 12836 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:40:03.700906 12934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:03.701067 12836 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:03.701138 12836 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:40:03.701164 12836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-2/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-2/wal
uuid: "1a6c09a7da984a169cf84a7187990d2b"
format_stamp: "Formatted at 2026-06-19 02:40:03 on dist-test-slave-zpbg"
server_key: "15ca7cefb56df29ce6dad1570318830b"
server_key_iv: "bb6a7526a460b92dbe62301766a459b4"
server_key_version: "encryptionkey@0"
I20260619 02:40:03.701856 12836 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:40:03.703302 12836 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260619 02:40:03.703369 12836 log.cc:1196] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260619 02:40:03.703564 12836 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b: Deleting consensus metadata
I20260619 02:40:03.703747 12836 master_runner.cc:315] Copying system tablet from 127.5.241.126:45231
I20260619 02:40:03.704334 12836 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b: tablet copy: Beginning tablet copy session from remote peer at address 127.5.241.126:45231
I20260619 02:40:03.711603 12224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:03.704673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51038 (local address 127.5.241.126:45231)
0619 02:40:03.704718 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:03.704720 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:03.704839 (+ 119us) server_negotiation.cc:407] Connection header received
0619 02:40:03.704933 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:03.704934 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:03.704944 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:03.704973 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:03.705352 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.706112 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.707135 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.707441 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.707484 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:03.708663 (+ 1179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:03.708665 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:03.708665 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:03.708673 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:03.710388 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:03.710819 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:03.710821 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:03.710821 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:03.710875 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:03.711070 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:03.711071 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:03.711072 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:03.711197 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:03.711239 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:03.711433 (+ 194us) server_negotiation.cc:299] Negotiation successful
0619 02:40:03.711509 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:40:03.712163 12940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:03.704615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45231 (local address 127.0.0.1:51038)
0619 02:40:03.704765 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:03.704776 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:03.704863 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:03.705031 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:03.705032 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:03.705227 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:03.705275 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:03.705277 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.706205 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.706206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:03.707038 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:03.707041 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.707571 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.707572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:03.707624 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:03.708165 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:03.708171 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:03.708563 (+ 392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:03.710482 (+ 1919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:03.710483 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:03.710484 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:03.710690 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:03.710948 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:03.710949 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:03.710949 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:03.711003 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:03.711283 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:03.711284 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:03.711327 (+ 43us) client_negotiation.cc:769] Sending connection context
0619 02:40:03.711967 (+ 640us) client_negotiation.cc:240] Negotiation successful
0619 02:40:03.712072 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":61,"threads_started":1}
I20260619 02:40:03.712699 12205 tablet_copy_service.cc:140] P 0305088fb8294ea485bdf6424e661096: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 1a6c09a7da984a169cf84a7187990d2b ({username='oryx', principal='oryx/127.5.241.124@KRBTEST.COM'} at 127.0.0.1:51038)
I20260619 02:40:03.712759 12205 tablet_copy_service.cc:161] P 0305088fb8294ea485bdf6424e661096: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 1a6c09a7da984a169cf84a7187990d2b at {username='oryx', principal='oryx/127.5.241.124@KRBTEST.COM'} at 127.0.0.1:51038: session id = 1a6c09a7da984a169cf84a7187990d2b-00000000000000000000000000000000
I20260619 02:40:03.713373 12205 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096: Tablet Copy: opened 0 blocks and 1 log segments
I20260619 02:40:03.713860 12836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260619 02:40:03.715215 12836 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b: tablet copy: Starting download of 0 data blocks...
I20260619 02:40:03.715356 12836 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b: tablet copy: Starting download of 1 WAL segments...
I20260619 02:40:03.716159 12836 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260619 02:40:03.717070 12205 tablet_copy_service.cc:342] P 0305088fb8294ea485bdf6424e661096: Request end of tablet copy session 1a6c09a7da984a169cf84a7187990d2b-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.5.241.124@KRBTEST.COM'} at 127.0.0.1:51038
I20260619 02:40:03.717121 12205 tablet_copy_service.cc:433] P 0305088fb8294ea485bdf6424e661096: ending tablet copy session 1a6c09a7da984a169cf84a7187990d2b-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 1a6c09a7da984a169cf84a7187990d2b
I20260619 02:40:03.719686 12836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:03.720969 12941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:03.723716 12942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:03.724256 12836 server_base.cc:1034] running on GCE node
W20260619 02:40:03.730928 12944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:03.731139 12836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:03.731298 12836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:03.732419 12836 hybrid_clock.cc:648] HybridClock initialized: now 1781836803732396 us; error 44 us; skew 500 ppm
Jun 19 02:40:03 dist-test-slave-zpbg krb5kdc[12140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836803, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.241.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:03.734005 12836 init.cc:377] Logged in from keytab as oryx/127.5.241.124@KRBTEST.COM (short username oryx)
I20260619 02:40:03.734277 12836 webserver.cc:460] Webserver started at http://127.5.241.124:43977/ using document root <none> and password file <none>
I20260619 02:40:03.734416 12836 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:03.734455 12836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:03.735244 12836 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:40:03.735872 12951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:03.736044 12836 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:40:03.736102 12836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-2/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-2/wal
uuid: "1a6c09a7da984a169cf84a7187990d2b"
format_stamp: "Formatted at 2026-06-19 02:40:03 on dist-test-slave-zpbg"
server_key: "15ca7cefb56df29ce6dad1570318830b"
server_key_iv: "bb6a7526a460b92dbe62301766a459b4"
server_key_version: "encryptionkey@0"
I20260619 02:40:03.736168 12836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:03.751852 12836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:03.753778 12836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:03.756908 12836 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.124:36925
I20260619 02:40:03.757273 13003 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.124:36925 every 8 connection(s)
I20260619 02:40:03.757308 12836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/master-2/data/info.pb
I20260619 02:40:03.758271 13004 sys_catalog.cc:263] Verifying existing consensus state
I20260619 02:40:03.758882 13004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b: Bootstrap starting.
I20260619 02:40:03.762272 13004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:40:03.762579 13004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b: Bootstrap complete.
I20260619 02:40:03.762892 13004 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } } peers { permanent_uuid: "56c49503edc647499d6dcab7b82fcde9" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 38543 } attrs { promote: false } } peers { permanent_uuid: "1a6c09a7da984a169cf84a7187990d2b" member_type: NON_VOTER last_known_addr { host: "127.5.241.124" port: 36925 } attrs { promote: true } }
I20260619 02:40:03.763063 13004 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1a6c09a7da984a169cf84a7187990d2b, State: Initialized, Role: LEARNER
I20260619 02:40:03.763144 13004 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } } peers { permanent_uuid: "56c49503edc647499d6dcab7b82fcde9" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 38543 } attrs { promote: false } } peers { permanent_uuid: "1a6c09a7da984a169cf84a7187990d2b" member_type: NON_VOTER last_known_addr { host: "127.5.241.124" port: 36925 } attrs { promote: true } }
I20260619 02:40:03.763418 13004 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:40:03.763449 13007 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } } peers { permanent_uuid: "56c49503edc647499d6dcab7b82fcde9" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 38543 } attrs { promote: false } } peers { permanent_uuid: "1a6c09a7da984a169cf84a7187990d2b" member_type: NON_VOTER last_known_addr { host: "127.5.241.124" port: 36925 } attrs { promote: true } } }
I20260619 02:40:03.763564 13007 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [sys.catalog]: This master's current role is: LEARNER
I20260619 02:40:03.765069 13018 catalog_manager.cc:1259] Loaded cluster ID: c32bc2d7e2e547d6adb2678ff8bbe53c
I20260619 02:40:03.765115 13018 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b: loading cluster ID for follower catalog manager: success
I20260619 02:40:03.766323 13018 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b: acquiring CA information for follower catalog manager: success
I20260619 02:40:03.766615 13018 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260619 02:40:03.767969 12929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:03.768200 12929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:03.768265 12929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:03.769526 12929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:03.769567 12929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:03.769595 12929 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.65
I20260619 02:40:03.771205 12929 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:44915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.65.keytab
--principal=oryx/127.5.241.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.65:44389
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.241.65
--webserver_port=45201
--tserver_master_addrs=127.5.241.126:45231,127.5.241.125:38543,127.5.241.124:36925
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12929
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.65
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:40:03.771452 12929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:03.771698 12929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:03.776661 13023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:03.776695 13024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:03.776685 13026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:03.777400 12929 server_base.cc:1034] running on GCE node
I20260619 02:40:03.777601 12929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:03.777853 12929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:03.779038 12929 hybrid_clock.cc:648] HybridClock initialized: now 1781836803779016 us; error 33 us; skew 500 ppm
Jun 19 02:40:03 dist-test-slave-zpbg krb5kdc[12140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836803, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.241.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:03.781821 12929 init.cc:377] Logged in from keytab as oryx/127.5.241.65@KRBTEST.COM (short username oryx)
I20260619 02:40:03.782338 12929 webserver.cc:460] Webserver started at http://127.5.241.65:45201/ using document root <none> and password file <none>
I20260619 02:40:03.782577 12929 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:03.782658 12929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:03.784139 12929 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:40:03.784869 13033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:03.785070 12929 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:40:03.785166 12929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-0/wal
uuid: "e90938d5560646d08f1a8872b88e0c40"
format_stamp: "Formatted at 2026-06-19 02:40:02 on dist-test-slave-zpbg"
server_key: "f1e7944be6dcdf6ecc04296a6b03e6a2"
server_key_iv: "ed188944cafe9d2661c55c074f3c1d6f"
server_key_version: "encryptionkey@0"
I20260619 02:40:03.785553 12929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:03.803586 12929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:03.805648 12929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:03.805816 12929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:03.806068 12929 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:40:03.806407 12929 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:40:03.806466 12929 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:03.806509 12929 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:40:03.806576 12929 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:03.812644 12929 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.65:44389
I20260619 02:40:03.812736 13146 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.65:44389 every 8 connection(s)
I20260619 02:40:03.813092 12929 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
Jun 19 02:40:03 dist-test-slave-zpbg krb5kdc[12140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836803, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.241.65@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM
I20260619 02:40:03.820570 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 12929
I20260619 02:40:03.820724 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 12381
Jun 19 02:40:03 dist-test-slave-zpbg krb5kdc[12140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836803, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.241.65@KRBTEST.COM for oryx/127.5.241.124@KRBTEST.COM
Jun 19 02:40:03 dist-test-slave-zpbg krb5kdc[12140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836803, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.241.65@KRBTEST.COM for oryx/127.5.241.125@KRBTEST.COM
I20260619 02:40:03.826460 12821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:03.813896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:39753 (local address 127.5.241.125:38543)
0619 02:40:03.813949 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:03.813951 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:03.814258 (+ 307us) server_negotiation.cc:407] Connection header received
0619 02:40:03.814745 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:03.814747 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:03.814759 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:03.814864 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:03.815835 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.816574 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.817737 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.818023 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.818084 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:03.822992 (+ 4908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:03.822994 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:03.822994 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:03.823003 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:03.824803 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:03.825363 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:03.825364 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:03.825364 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:03.825422 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:03.825671 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:03.825672 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:03.825673 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:03.825841 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:03.825928 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:03.826232 (+ 304us) server_negotiation.cc:299] Negotiation successful
0619 02:40:03.826337 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 02:40:03.826722 13153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:03.813775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.125:38543 (local address 127.5.241.65:39753)
0619 02:40:03.814192 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:03.814204 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:03.814647 (+ 443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:03.814942 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:03.814946 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:03.815480 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:03.815668 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:03.815673 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.816647 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.816648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:03.817657 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:03.817659 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.818133 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.818134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:03.818186 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:03.818992 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:03.818997 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:03.822871 (+ 3874us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:03.824904 (+ 2033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:03.824905 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:03.824906 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:03.825277 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:03.825490 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:03.825490 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:03.825491 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:03.825593 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:03.825918 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:03.825925 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:03.826084 (+ 159us) client_negotiation.cc:769] Sending connection context
0619 02:40:03.826203 (+ 119us) client_negotiation.cc:240] Negotiation successful
0619 02:40:03.826514 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"mutex_wait_us":444,"thread_start_us":65,"threads_started":1}
I20260619 02:40:03.826941 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.66:36493
--local_ip_for_outbound_sockets=127.5.241.66
--tserver_master_addrs=127.5.241.126:45231
--webserver_port=44283
--webserver_interface=127.5.241.66
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.66.keytab
--principal=oryx/127.5.241.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:44915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.5.241.126:45231,127.5.241.125:38543
--tserver_master_addrs=127.5.241.126:45231,127.5.241.125:38543,127.5.241.124:36925 with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:40:03.827083 13154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:03.814083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:35569 (local address 127.5.241.124:36925)
0619 02:40:03.814276 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:03.814278 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:03.814286 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:40:03.814982 (+ 696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:03.814983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:03.814998 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:03.815067 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:03.815776 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.816776 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.817582 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.817873 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.817932 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:03.822603 (+ 4671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:03.822605 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:03.822605 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:03.822615 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:03.824660 (+ 2045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:03.825302 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:03.825303 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:03.825304 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:03.825373 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:03.825694 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:03.825695 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:03.825696 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:03.825852 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:03.825920 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:03.826864 (+ 944us) server_negotiation.cc:299] Negotiation successful
0619 02:40:03.826956 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":44,"threads_started":1}
I20260619 02:40:03.827325 13152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:03.813774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.124:36925 (local address 127.5.241.65:35569)
0619 02:40:03.814015 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:03.814059 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:03.814631 (+ 572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:03.815086 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:03.815086 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:03.815474 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:03.815668 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:03.815673 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.816860 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.816861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:03.817486 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:03.817488 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.817995 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.817996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:03.818155 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:03.818867 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:03.818877 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:03.822459 (+ 3582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:03.824774 (+ 2315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:03.824782 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:03.824785 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:03.825193 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:03.825450 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:03.825451 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:03.825451 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:03.825596 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:03.825948 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:03.825949 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:03.826087 (+ 138us) client_negotiation.cc:769] Sending connection context
0619 02:40:03.826203 (+ 116us) client_negotiation.cc:240] Negotiation successful
0619 02:40:03.826425 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"mutex_wait_us":14,"thread_start_us":86,"threads_started":1}
I20260619 02:40:03.827642 13149 heartbeater.cc:347] Connected to a master server at 127.5.241.125:38543
I20260619 02:40:03.827759 13149 heartbeater.cc:464] Registering TS with master...
I20260619 02:40:03.827965 13149 heartbeater.cc:511] Master 127.5.241.125:38543 requested a full tablet report, sending...
I20260619 02:40:03.828830 13147 heartbeater.cc:347] Connected to a master server at 127.5.241.124:36925
I20260619 02:40:03.828895 13147 heartbeater.cc:464] Registering TS with master...
I20260619 02:40:03.829057 13147 heartbeater.cc:511] Master 127.5.241.124:36925 requested a full tablet report, sending...
I20260619 02:40:03.829679 12782 ts_manager.cc:194] Registered new tserver with Master: e90938d5560646d08f1a8872b88e0c40 (127.5.241.65:44389)
I20260619 02:40:03.829816 12968 ts_manager.cc:194] Registered new tserver with Master: e90938d5560646d08f1a8872b88e0c40 (127.5.241.65:44389)
I20260619 02:40:03.832407 13151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:03.813809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45231 (local address 127.5.241.65:39463)
0619 02:40:03.814155 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:03.814165 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:03.814637 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:03.815270 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:03.815271 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:03.815470 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:03.815667 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:03.815671 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.816465 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.816466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:03.817657 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:03.817659 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.818053 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.818054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:03.818153 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:03.818884 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:03.818890 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:03.828684 (+ 9794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:03.831129 (+ 2445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:03.831130 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:03.831130 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:03.831386 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:03.831604 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:03.831605 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:03.831606 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:03.831677 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:03.832005 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:03.832006 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:03.832050 (+ 44us) client_negotiation.cc:769] Sending connection context
0619 02:40:03.832146 (+ 96us) client_negotiation.cc:240] Negotiation successful
0619 02:40:03.832265 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"mutex_wait_us":31,"thread_start_us":106,"threads_started":1}
I20260619 02:40:03.832450 12224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:03.814017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:39463 (local address 127.5.241.126:45231)
0619 02:40:03.814192 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:03.814194 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:03.814208 (+ 14us) server_negotiation.cc:407] Connection header received
0619 02:40:03.814829 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:03.814830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:03.814841 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:03.814923 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:03.815852 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.816360 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.817766 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.817978 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.818054 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:03.828827 (+ 10773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:03.828829 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:03.828830 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:03.828839 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:03.831048 (+ 2209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:03.831466 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:03.831467 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:03.831467 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:03.831523 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:03.831746 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:03.831747 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:03.831748 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:03.831917 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:03.831980 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:03.832217 (+ 237us) server_negotiation.cc:299] Negotiation successful
0619 02:40:03.832317 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146}
I20260619 02:40:03.833128 13148 heartbeater.cc:347] Connected to a master server at 127.5.241.126:45231
I20260619 02:40:03.833194 13148 heartbeater.cc:464] Registering TS with master...
I20260619 02:40:03.833273 13148 heartbeater.cc:511] Master 127.5.241.126:45231 requested a full tablet report, sending...
I20260619 02:40:03.833549 12185 ts_manager.cc:194] Re-registered known tserver with Master: e90938d5560646d08f1a8872b88e0c40 (127.5.241.65:44389)
I20260619 02:40:03.833940 12185 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.241.65@KRBTEST.COM'} at 127.5.241.65:39463
W20260619 02:40:03.913071 13155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:03.913357 13155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:03.913399 13155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:03.915170 13155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:03.915241 13155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:03.915290 13155 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.66
I20260619 02:40:03.916966 13155 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:44915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.66.keytab
--principal=oryx/127.5.241.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.66:36493
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.241.66
--webserver_port=44283
--tserver_master_addrs=127.5.241.126:45231,127.5.241.125:38543,127.5.241.124:36925
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13155
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.66
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:40:03.917261 13155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:03.917533 13155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:03.923105 13163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:03.923091 13155 server_base.cc:1034] running on GCE node
W20260619 02:40:03.923086 13161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:03.923256 13160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:03.923601 13155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:03.923817 13155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:03.924954 13155 hybrid_clock.cc:648] HybridClock initialized: now 1781836803924937 us; error 37 us; skew 500 ppm
Jun 19 02:40:03 dist-test-slave-zpbg krb5kdc[12140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836803, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.241.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:03.927799 13155 init.cc:377] Logged in from keytab as oryx/127.5.241.66@KRBTEST.COM (short username oryx)
I20260619 02:40:03.928274 13155 webserver.cc:460] Webserver started at http://127.5.241.66:44283/ using document root <none> and password file <none>
I20260619 02:40:03.928498 13155 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:03.928553 13155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:03.929977 13155 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:40:03.930552 13170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:03.930739 13155 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:40:03.930873 13155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-1/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-1/wal
uuid: "f277ace414904d888dfcd10ba52bc2c5"
format_stamp: "Formatted at 2026-06-19 02:40:02 on dist-test-slave-zpbg"
server_key: "9c663800930cb639b2182a72b1e14e7e"
server_key_iv: "8ca9cbb3d5e947244c630e7911cabc4c"
server_key_version: "encryptionkey@0"
I20260619 02:40:03.931272 13155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:03.972796 13155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:03.975186 13155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:03.975348 13155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:03.975605 13155 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:40:03.975911 13155 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:40:03.975965 13155 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:03.976006 13155 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:40:03.976032 13155 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:03.981904 13155 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.66:36493
I20260619 02:40:03.981945 13283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.66:36493 every 8 connection(s)
I20260619 02:40:03.982281 13155 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
I20260619 02:40:03.983758 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 13155
I20260619 02:40:03.983922 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 12515
Jun 19 02:40:03 dist-test-slave-zpbg krb5kdc[12140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836803, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.241.66@KRBTEST.COM for oryx/127.5.241.125@KRBTEST.COM
Jun 19 02:40:03 dist-test-slave-zpbg krb5kdc[12140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836803, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.241.66@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM
I20260619 02:40:03.990983 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.67:45923
--local_ip_for_outbound_sockets=127.5.241.67
--tserver_master_addrs=127.5.241.126:45231
--webserver_port=40553
--webserver_interface=127.5.241.67
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.67.keytab
--principal=oryx/127.5.241.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:44915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.5.241.126:45231,127.5.241.125:38543
--tserver_master_addrs=127.5.241.126:45231,127.5.241.125:38543,127.5.241.124:36925 with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 02:40:03 dist-test-slave-zpbg krb5kdc[12140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836803, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.241.66@KRBTEST.COM for oryx/127.5.241.124@KRBTEST.COM
I20260619 02:40:03.994899 12821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:03.983529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:48259 (local address 127.5.241.125:38543)
0619 02:40:03.983582 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:03.983584 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:03.983611 (+ 27us) server_negotiation.cc:407] Connection header received
0619 02:40:03.984127 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:03.984129 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:03.984140 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:03.984169 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:03.984979 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.985682 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.987647 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.987953 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.987994 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:03.991333 (+ 3339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:03.991335 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:03.991335 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:03.991343 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:03.993045 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:03.993772 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:03.993773 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:03.993774 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:03.993832 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:03.994113 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:03.994114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:03.994115 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:03.994250 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:03.994294 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:03.994675 (+ 381us) server_negotiation.cc:299] Negotiation successful
0619 02:40:03.994759 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 02:40:03.995432 13290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:03.983425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.125:38543 (local address 127.5.241.66:48259)
0619 02:40:03.983558 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:03.983567 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:03.984050 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:03.984608 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:03.984609 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:03.984825 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:03.984896 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:03.984898 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.985777 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.985778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:03.987542 (+ 1764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:03.987545 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.988085 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.988086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:03.988244 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:03.988935 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:03.988943 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:03.991249 (+ 2306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:03.993351 (+ 2102us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:03.993358 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:03.993360 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:03.993674 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:03.993905 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:03.993906 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:03.993906 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:03.994033 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:03.994340 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:03.994346 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:03.994520 (+ 174us) client_negotiation.cc:769] Sending connection context
0619 02:40:03.994989 (+ 469us) client_negotiation.cc:240] Negotiation successful
0619 02:40:03.995176 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":59,"thread_start_us":82,"threads_started":1}
I20260619 02:40:03.995674 13154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:03.983010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:46295 (local address 127.5.241.124:36925)
0619 02:40:03.983054 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:03.983057 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:03.983440 (+ 383us) server_negotiation.cc:407] Connection header received
0619 02:40:03.985777 (+ 2337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:03.985778 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:03.985794 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:03.985845 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:03.986225 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.987070 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.988213 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.989160 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.989227 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:03.992551 (+ 3324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:03.992553 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:03.992554 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:03.992562 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:03.994253 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:03.994599 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:03.994600 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:03.994600 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:03.994658 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:03.994866 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:03.994877 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:03.994878 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:03.995005 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:03.995065 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:03.995485 (+ 420us) server_negotiation.cc:299] Negotiation successful
0619 02:40:03.995574 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 02:40:03.995432 13288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:03.983019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.124:36925 (local address 127.5.241.66:46295)
0619 02:40:03.983350 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:03.983367 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:03.983930 (+ 563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:03.985889 (+ 1959us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:03.985890 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:03.986067 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:03.986132 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:03.986134 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.987158 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.987159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:03.988120 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:03.988122 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.989280 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.989281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:03.989355 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:03.989789 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:03.989794 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:03.992044 (+ 2250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:03.994332 (+ 2288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:03.994333 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:03.994334 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:03.994519 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:03.994723 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:03.994723 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:03.994724 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:03.994799 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:03.995089 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:03.995090 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:03.995144 (+ 54us) client_negotiation.cc:769] Sending connection context
0619 02:40:03.995206 (+ 62us) client_negotiation.cc:240] Negotiation successful
0619 02:40:03.995286 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"mutex_wait_us":2,"thread_start_us":92,"threads_started":1}
I20260619 02:40:03.995817 12224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:03.983009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:33593 (local address 127.5.241.126:45231)
0619 02:40:03.983057 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:03.983059 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:03.983326 (+ 267us) server_negotiation.cc:407] Connection header received
0619 02:40:03.984039 (+ 713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:03.984041 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:03.984055 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:03.984083 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:03.984861 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.985667 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.987648 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.987950 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.987994 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:03.991956 (+ 3962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:03.991957 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:03.991958 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:03.991966 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:03.994478 (+ 2512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:03.994935 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:03.994936 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:03.994936 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:03.995002 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:03.995200 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:03.995201 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:03.995201 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:03.995342 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:03.995400 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:03.995606 (+ 206us) server_negotiation.cc:299] Negotiation successful
0619 02:40:03.995680 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 02:40:03.995868 13286 heartbeater.cc:347] Connected to a master server at 127.5.241.125:38543
I20260619 02:40:03.995954 13286 heartbeater.cc:464] Registering TS with master...
I20260619 02:40:03.996147 13289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:03.983019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45231 (local address 127.5.241.66:33593)
0619 02:40:03.983195 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:03.983229 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:03.983915 (+ 686us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:03.984209 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:03.984212 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:03.984564 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:03.984758 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:03.984762 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.985782 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.985783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:03.987542 (+ 1759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:03.987545 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:03.988086 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:03.988087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:03.988244 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:03.989016 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:03.989021 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:03.991844 (+ 2823us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:03.994571 (+ 2727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:03.994572 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:03.994572 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:03.994838 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:03.995082 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:03.995083 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:03.995083 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:03.995137 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:03.995436 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:03.995436 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:03.995481 (+ 45us) client_negotiation.cc:769] Sending connection context
0619 02:40:03.995935 (+ 454us) client_negotiation.cc:240] Negotiation successful
0619 02:40:03.996041 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":54,"spinlock_wait_cycles":7552,"thread_start_us":91,"threads_started":1}
I20260619 02:40:03.996153 13286 heartbeater.cc:511] Master 127.5.241.125:38543 requested a full tablet report, sending...
I20260619 02:40:03.996299 13284 heartbeater.cc:347] Connected to a master server at 127.5.241.124:36925
I20260619 02:40:03.996328 13284 heartbeater.cc:464] Registering TS with master...
I20260619 02:40:03.996381 13284 heartbeater.cc:511] Master 127.5.241.124:36925 requested a full tablet report, sending...
I20260619 02:40:03.996459 13285 heartbeater.cc:347] Connected to a master server at 127.5.241.126:45231
I20260619 02:40:03.996505 13285 heartbeater.cc:464] Registering TS with master...
I20260619 02:40:03.996583 13285 heartbeater.cc:511] Master 127.5.241.126:45231 requested a full tablet report, sending...
I20260619 02:40:03.996618 12968 ts_manager.cc:194] Registered new tserver with Master: f277ace414904d888dfcd10ba52bc2c5 (127.5.241.66:36493)
I20260619 02:40:03.996613 12782 ts_manager.cc:194] Registered new tserver with Master: f277ace414904d888dfcd10ba52bc2c5 (127.5.241.66:36493)
I20260619 02:40:03.996820 12185 ts_manager.cc:194] Re-registered known tserver with Master: f277ace414904d888dfcd10ba52bc2c5 (127.5.241.66:36493)
I20260619 02:40:03.997363 12185 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.241.66@KRBTEST.COM'} at 127.5.241.66:33593
W20260619 02:40:04.075593 13291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:04.076007 13291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:04.076041 13291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:04.077313 13291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:04.077358 13291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:04.077387 13291 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.67
I20260619 02:40:04.079007 13291 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:44915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.67.keytab
--principal=oryx/127.5.241.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.67:45923
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.241.67
--webserver_port=40553
--tserver_master_addrs=127.5.241.126:45231,127.5.241.125:38543,127.5.241.124:36925
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13291
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.67
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:40:04.079249 13291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:04.079509 13291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:04.084721 13297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:04.084712 13299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:04.084745 13291 server_base.cc:1034] running on GCE node
W20260619 02:40:04.084713 13296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:04.085124 13291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:04.085398 13291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:04.086557 13291 hybrid_clock.cc:648] HybridClock initialized: now 1781836804086542 us; error 31 us; skew 500 ppm
Jun 19 02:40:04 dist-test-slave-zpbg krb5kdc[12140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836804, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.241.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:04.089131 13291 init.cc:377] Logged in from keytab as oryx/127.5.241.67@KRBTEST.COM (short username oryx)
I20260619 02:40:04.089568 13291 webserver.cc:460] Webserver started at http://127.5.241.67:40553/ using document root <none> and password file <none>
I20260619 02:40:04.089759 13291 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:04.089812 13291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:04.091284 13291 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:40:04.091909 13306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:04.092072 13291 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 02:40:04.092165 13291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-2/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-2/wal
uuid: "39f073a2df8741218340f6e1a84deb6a"
format_stamp: "Formatted at 2026-06-19 02:40:03 on dist-test-slave-zpbg"
server_key: "47b1e7932ebd7465390881c21c93c195"
server_key_iv: "781fc4449303bc4a1e7d812b8940f70f"
server_key_version: "encryptionkey@0"
I20260619 02:40:04.092509 13291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:04.108852 13291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:04.111342 13291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:04.111510 13291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:04.111752 13291 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:40:04.112063 13291 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:40:04.112115 13291 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:04.112159 13291 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:40:04.112186 13291 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:04.117832 13291 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.67:45923
I20260619 02:40:04.117893 13419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.67:45923 every 8 connection(s)
I20260619 02:40:04.118256 13291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
Jun 19 02:40:04 dist-test-slave-zpbg krb5kdc[12140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836804, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.241.67@KRBTEST.COM for oryx/127.5.241.125@KRBTEST.COM
Jun 19 02:40:04 dist-test-slave-zpbg krb5kdc[12140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836804, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.241.67@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM
Jun 19 02:40:04 dist-test-slave-zpbg krb5kdc[12140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836804, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.241.67@KRBTEST.COM for oryx/127.5.241.124@KRBTEST.COM
I20260619 02:40:04.126654 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 13291
I20260619 02:40:04.130322 13424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:04.118995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.125:38543 (local address 127.5.241.67:36369)
0619 02:40:04.119330 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:04.119339 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:04.119808 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:04.120059 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:04.120063 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:04.120533 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:04.120739 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:04.120744 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:04.121523 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:04.121524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:04.122552 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:04.122554 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:04.123096 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:04.123097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:04.123226 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:04.123862 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:04.123867 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:04.125471 (+ 1604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:04.127474 (+ 2003us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:04.127480 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:04.127483 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:04.127841 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:04.128077 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:04.128078 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:04.128078 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:04.128231 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:04.129588 (+ 1357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:04.129595 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:04.129770 (+ 175us) client_negotiation.cc:769] Sending connection context
0619 02:40:04.129891 (+ 121us) client_negotiation.cc:240] Negotiation successful
0619 02:40:04.130086 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"mutex_wait_us":29,"thread_start_us":82,"threads_started":1}
I20260619 02:40:04.131137 13425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:04.118995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.124:36925 (local address 127.5.241.67:41353)
0619 02:40:04.119179 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:04.119223 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:04.119795 (+ 572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:04.120060 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:04.120063 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:04.120533 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:04.120740 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:04.120745 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:04.121567 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:04.121568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:04.122616 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:04.122619 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:04.123101 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:04.123102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:04.123224 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:04.123823 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:04.123833 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:04.126411 (+ 2578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:04.129931 (+ 3520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:04.129931 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:04.129932 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:04.130161 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:04.130356 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:04.130357 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:04.130357 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:04.130413 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:04.130665 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:04.130666 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:04.130706 (+ 40us) client_negotiation.cc:769] Sending connection context
0619 02:40:04.130801 (+ 95us) client_negotiation.cc:240] Negotiation successful
0619 02:40:04.130897 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":25,"thread_start_us":103,"threads_started":1}
I20260619 02:40:04.131176 13154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:04.119071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:41353 (local address 127.5.241.124:36925)
0619 02:40:04.119115 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:04.119118 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:04.119321 (+ 203us) server_negotiation.cc:407] Connection header received
0619 02:40:04.119908 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:04.119909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:04.119934 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:04.119984 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:04.120903 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:04.121492 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:04.122687 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:04.123014 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:04.123099 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:04.127883 (+ 4784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:04.127885 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:04.127885 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:04.127893 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:04.129842 (+ 1949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:04.130231 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:04.130232 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:04.130232 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:04.130287 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:04.130476 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:04.130477 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:04.130477 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:04.130588 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:04.130641 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:04.131012 (+ 371us) server_negotiation.cc:299] Negotiation successful
0619 02:40:04.131089 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 02:40:04.131542 13420 heartbeater.cc:347] Connected to a master server at 127.5.241.124:36925
I20260619 02:40:04.131633 13420 heartbeater.cc:464] Registering TS with master...
I20260619 02:40:04.131836 13420 heartbeater.cc:511] Master 127.5.241.124:36925 requested a full tablet report, sending...
I20260619 02:40:04.132174 13426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:04.118995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45231 (local address 127.5.241.67:51709)
0619 02:40:04.120288 (+ 1293us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:04.120298 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:04.120368 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:04.120515 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:04.120515 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:04.120666 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:04.120740 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:04.120745 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:04.122421 (+ 1676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:04.122422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:04.123224 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:04.123225 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:04.123656 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:04.123657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:04.123724 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:04.124035 (+ 311us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:04.124040 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:04.126198 (+ 2158us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:04.129284 (+ 3086us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:04.129285 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:04.129286 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:04.129518 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:04.130950 (+ 1432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:04.130951 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:04.130952 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:04.131028 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:04.131847 (+ 819us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:04.131848 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:04.131887 (+ 39us) client_negotiation.cc:769] Sending connection context
0619 02:40:04.131954 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 02:40:04.132056 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1272,"mutex_wait_us":350,"thread_start_us":68,"threads_started":1}
I20260619 02:40:04.132292 12968 ts_manager.cc:194] Registered new tserver with Master: 39f073a2df8741218340f6e1a84deb6a (127.5.241.67:45923)
I20260619 02:40:04.132782 12821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:04.119064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:36369 (local address 127.5.241.125:38543)
0619 02:40:04.119114 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:04.119116 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:04.119420 (+ 304us) server_negotiation.cc:407] Connection header received
0619 02:40:04.119901 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:04.119902 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:04.119913 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:04.119985 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:04.120909 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:04.121407 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:04.122646 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:04.122972 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:04.123015 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:04.125581 (+ 2566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:04.125582 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:04.125582 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:04.125589 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:04.127413 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:04.127909 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:04.127911 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:04.127911 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:04.127975 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:04.128310 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:04.128311 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:04.128311 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:04.128453 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:04.128546 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:04.132587 (+ 4041us) server_negotiation.cc:299] Negotiation successful
0619 02:40:04.132673 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 02:40:04.133082 13422 heartbeater.cc:347] Connected to a master server at 127.5.241.125:38543
I20260619 02:40:04.133134 13422 heartbeater.cc:464] Registering TS with master...
I20260619 02:40:04.133198 13422 heartbeater.cc:511] Master 127.5.241.125:38543 requested a full tablet report, sending...
I20260619 02:40:04.133224 12224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:04.119230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:51709 (local address 127.5.241.126:45231)
0619 02:40:04.119294 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:04.119296 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:04.120358 (+ 1062us) server_negotiation.cc:407] Connection header received
0619 02:40:04.120421 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:04.120422 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:04.120432 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:04.120495 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:04.121593 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:04.122341 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:04.123300 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:04.123565 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:04.123654 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:04.126289 (+ 2635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:04.126290 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:04.126291 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:04.126299 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:04.129192 (+ 2893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:04.130814 (+ 1622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:04.130815 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:04.130815 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:04.130874 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:04.131586 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:04.131587 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:04.131587 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:04.131714 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:04.131776 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:04.133045 (+ 1269us) server_negotiation.cc:299] Negotiation successful
0619 02:40:04.133135 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 02:40:04.133486 13421 heartbeater.cc:347] Connected to a master server at 127.5.241.126:45231
I20260619 02:40:04.133489 12782 ts_manager.cc:194] Registered new tserver with Master: 39f073a2df8741218340f6e1a84deb6a (127.5.241.67:45923)
I20260619 02:40:04.133533 13421 heartbeater.cc:464] Registering TS with master...
I20260619 02:40:04.133623 13421 heartbeater.cc:511] Master 127.5.241.126:45231 requested a full tablet report, sending...
I20260619 02:40:04.133864 12185 ts_manager.cc:194] Re-registered known tserver with Master: 39f073a2df8741218340f6e1a84deb6a (127.5.241.67:45923)
I20260619 02:40:04.134359 12185 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.241.67@KRBTEST.COM'} at 127.5.241.67:51709
I20260619 02:40:04.138315 13433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:04.127984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51048 (local address 127.5.241.126:45231)
0619 02:40:04.129139 (+ 1155us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:04.129142 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:04.129343 (+ 201us) server_negotiation.cc:407] Connection header received
0619 02:40:04.129364 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:04.129365 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:04.129377 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:04.129443 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:04.130140 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:04.130898 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:04.132206 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:04.132515 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:04.132558 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:04.134529 (+ 1971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:04.134530 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:04.134530 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:04.134537 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:04.136123 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:04.137023 (+ 900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:04.137024 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:04.137025 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:04.137089 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:04.137569 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:04.137570 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:04.137570 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:04.137708 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:04.137816 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:04.138109 (+ 293us) server_negotiation.cc:299] Negotiation successful
0619 02:40:04.138195 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1125,"thread_start_us":77,"threads_started":1}
I20260619 02:40:04.139334 13434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:04.128115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60386 (local address 127.5.241.124:36925)
0619 02:40:04.128560 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:04.128562 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:04.128570 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:40:04.128590 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:04.128591 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:04.128603 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:04.128677 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:04.129490 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:04.130270 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:04.131358 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:04.131643 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:04.131685 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:04.136301 (+ 4616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:04.136302 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:04.136302 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:04.136309 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:04.138054 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:04.138456 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:04.138457 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:04.138457 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:04.138520 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:04.138714 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:04.138715 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:04.138715 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:04.138869 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:04.138932 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:04.139137 (+ 205us) server_negotiation.cc:299] Negotiation successful
0619 02:40:04.139226 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":72,"threads_started":1}
I20260619 02:40:04.140625 12821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:04.128435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37464 (local address 127.5.241.125:38543)
0619 02:40:04.132880 (+ 4445us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:04.132882 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:04.132889 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:40:04.132908 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:04.132908 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:04.132918 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:04.132978 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:04.134472 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:04.135288 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:04.136078 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:04.136365 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:04.136448 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:04.137635 (+ 1187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:04.137636 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:04.137637 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:04.137643 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:04.139355 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:04.139852 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:04.139854 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:04.139854 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:04.139930 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:04.140132 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:04.140133 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:04.140133 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:04.140243 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:04.140339 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:04.140463 (+ 124us) server_negotiation.cc:299] Negotiation successful
0619 02:40:04.140529 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4430,"thread_start_us":83,"threads_started":1}
I20260619 02:40:04.139941 12185 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51048:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:40:04.141680 12185 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:40:04.148746 13442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:04.145299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.67:45923 (local address 127.0.0.1:39114)
0619 02:40:04.145470 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:04.145517 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:04.145565 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:04.145955 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:04.145956 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:04.145957 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:40:04.146010 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:04.146012 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:04.146825 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:04.146826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:04.147625 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:04.147627 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:04.148514 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:04.148515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:04.148575 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:04.148580 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 02:40:04.148634 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 02:40:04.148672 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":2,"thread_start_us":82,"threads_started":1}
I20260619 02:40:04.148759 13445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:04.145648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39114 (local address 127.5.241.67:45923)
0619 02:40:04.145796 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:04.145800 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:04.145810 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:40:04.145853 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:04.145854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:04.145867 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:04.145957 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:40:04.146065 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:04.146721 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:04.147718 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:04.148433 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:04.148505 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:04.148562 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:04.148645 (+ 83us) server_negotiation.cc:299] Negotiation successful
0619 02:40:04.148682 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":91,"threads_started":1}
I20260619 02:40:04.149562 12742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:04.145223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.66:36493 (local address 127.0.0.1:59488)
0619 02:40:04.145255 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:04.145266 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:04.145303 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:04.145955 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:04.145956 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:04.145958 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:40:04.146007 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:04.146009 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:04.147277 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:04.147278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:04.148235 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:04.148237 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:04.149306 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:04.149308 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:04.149365 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:04.149368 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:40:04.149440 (+ 72us) client_negotiation.cc:240] Negotiation successful
0619 02:40:04.149486 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16,"mutex_wait_us":59}
I20260619 02:40:04.149567 13444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:04.145438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59488 (local address 127.5.241.66:36493)
0619 02:40:04.145807 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:04.145817 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:04.145824 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:40:04.145856 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:04.145857 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:04.145869 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:04.145958 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:40:04.146088 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:04.147187 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:04.148329 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:04.149222 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:04.149293 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:04.149357 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:04.149454 (+ 97us) server_negotiation.cc:299] Negotiation successful
0619 02:40:04.149496 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":73,"threads_started":1}
I20260619 02:40:04.149497 13354 tablet_service.cc:1467] Processing CreateTablet for tablet d9051b6a90864b74a6322f368390390e (DEFAULT_TABLE table=test-table [id=61ad393d2ee5476499d4f8bd186027ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:40:04.149749 13441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:04.145320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:44389 (local address 127.0.0.1:39874)
0619 02:40:04.145638 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:04.145647 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:04.145690 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:04.146224 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:04.146225 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:04.146227 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:40:04.146259 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:04.146260 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:04.147479 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:04.147480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:04.148544 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:04.148547 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:04.149544 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:04.149545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:04.149604 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:04.149609 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 02:40:04.149639 (+ 30us) client_negotiation.cc:240] Negotiation successful
0619 02:40:04.149682 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":111,"threads_started":1}
I20260619 02:40:04.149861 13354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9051b6a90864b74a6322f368390390e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:04.150234 13443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:04.145418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39874 (local address 127.5.241.65:44389)
0619 02:40:04.146067 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:04.146071 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:04.146081 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:40:04.146117 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:04.146119 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:04.146133 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:04.146324 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:40:04.146388 (+ 64us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:04.147398 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:04.148631 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:04.149458 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:04.150017 (+ 559us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:04.150090 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:04.150116 (+ 26us) server_negotiation.cc:299] Negotiation successful
0619 02:40:04.150156 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":76,"threads_started":1}
I20260619 02:40:04.150264 13218 tablet_service.cc:1467] Processing CreateTablet for tablet d9051b6a90864b74a6322f368390390e (DEFAULT_TABLE table=test-table [id=61ad393d2ee5476499d4f8bd186027ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:40:04.150597 13218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9051b6a90864b74a6322f368390390e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:04.151441 13081 tablet_service.cc:1467] Processing CreateTablet for tablet d9051b6a90864b74a6322f368390390e (DEFAULT_TABLE table=test-table [id=61ad393d2ee5476499d4f8bd186027ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:40:04.151754 13081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9051b6a90864b74a6322f368390390e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:04.154016 13446 tablet_bootstrap.cc:492] T d9051b6a90864b74a6322f368390390e P 39f073a2df8741218340f6e1a84deb6a: Bootstrap starting.
I20260619 02:40:04.154228 13447 tablet_bootstrap.cc:492] T d9051b6a90864b74a6322f368390390e P f277ace414904d888dfcd10ba52bc2c5: Bootstrap starting.
I20260619 02:40:04.154757 13447 tablet_bootstrap.cc:654] T d9051b6a90864b74a6322f368390390e P f277ace414904d888dfcd10ba52bc2c5: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:04.155055 13448 tablet_bootstrap.cc:492] T d9051b6a90864b74a6322f368390390e P e90938d5560646d08f1a8872b88e0c40: Bootstrap starting.
I20260619 02:40:04.155118 13447 log.cc:824] T d9051b6a90864b74a6322f368390390e P f277ace414904d888dfcd10ba52bc2c5: Log is configured to *not* fsync() on all Append() calls
I20260619 02:40:04.155340 13446 tablet_bootstrap.cc:654] T d9051b6a90864b74a6322f368390390e P 39f073a2df8741218340f6e1a84deb6a: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:04.155643 13446 log.cc:824] T d9051b6a90864b74a6322f368390390e P 39f073a2df8741218340f6e1a84deb6a: Log is configured to *not* fsync() on all Append() calls
I20260619 02:40:04.155681 13448 tablet_bootstrap.cc:654] T d9051b6a90864b74a6322f368390390e P e90938d5560646d08f1a8872b88e0c40: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:04.155972 13447 tablet_bootstrap.cc:492] T d9051b6a90864b74a6322f368390390e P f277ace414904d888dfcd10ba52bc2c5: No bootstrap required, opened a new log
I20260619 02:40:04.155978 13448 log.cc:824] T d9051b6a90864b74a6322f368390390e P e90938d5560646d08f1a8872b88e0c40: Log is configured to *not* fsync() on all Append() calls
I20260619 02:40:04.156236 13447 ts_tablet_manager.cc:1397] T d9051b6a90864b74a6322f368390390e P f277ace414904d888dfcd10ba52bc2c5: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260619 02:40:04.156446 13446 tablet_bootstrap.cc:492] T d9051b6a90864b74a6322f368390390e P 39f073a2df8741218340f6e1a84deb6a: No bootstrap required, opened a new log
I20260619 02:40:04.156510 13446 ts_tablet_manager.cc:1397] T d9051b6a90864b74a6322f368390390e P 39f073a2df8741218340f6e1a84deb6a: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260619 02:40:04.158191 13447 raft_consensus.cc:348] T d9051b6a90864b74a6322f368390390e P f277ace414904d888dfcd10ba52bc2c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f277ace414904d888dfcd10ba52bc2c5" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36493 } } peers { permanent_uuid: "39f073a2df8741218340f6e1a84deb6a" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 45923 } } peers { permanent_uuid: "e90938d5560646d08f1a8872b88e0c40" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 44389 } }
I20260619 02:40:04.158366 13447 raft_consensus.cc:374] T d9051b6a90864b74a6322f368390390e P f277ace414904d888dfcd10ba52bc2c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:04.158408 13447 raft_consensus.cc:729] T d9051b6a90864b74a6322f368390390e P f277ace414904d888dfcd10ba52bc2c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f277ace414904d888dfcd10ba52bc2c5, State: Initialized, Role: FOLLOWER
I20260619 02:40:04.158366 13446 raft_consensus.cc:348] T d9051b6a90864b74a6322f368390390e P 39f073a2df8741218340f6e1a84deb6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f277ace414904d888dfcd10ba52bc2c5" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36493 } } peers { permanent_uuid: "39f073a2df8741218340f6e1a84deb6a" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 45923 } } peers { permanent_uuid: "e90938d5560646d08f1a8872b88e0c40" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 44389 } }
I20260619 02:40:04.158505 13446 raft_consensus.cc:374] T d9051b6a90864b74a6322f368390390e P 39f073a2df8741218340f6e1a84deb6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:04.158547 13446 raft_consensus.cc:729] T d9051b6a90864b74a6322f368390390e P 39f073a2df8741218340f6e1a84deb6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39f073a2df8741218340f6e1a84deb6a, State: Initialized, Role: FOLLOWER
I20260619 02:40:04.158525 13447 consensus_queue.cc:260] T d9051b6a90864b74a6322f368390390e P f277ace414904d888dfcd10ba52bc2c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f277ace414904d888dfcd10ba52bc2c5" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36493 } } peers { permanent_uuid: "39f073a2df8741218340f6e1a84deb6a" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 45923 } } peers { permanent_uuid: "e90938d5560646d08f1a8872b88e0c40" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 44389 } }
I20260619 02:40:04.158676 13446 consensus_queue.cc:260] T d9051b6a90864b74a6322f368390390e P 39f073a2df8741218340f6e1a84deb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f277ace414904d888dfcd10ba52bc2c5" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36493 } } peers { permanent_uuid: "39f073a2df8741218340f6e1a84deb6a" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 45923 } } peers { permanent_uuid: "e90938d5560646d08f1a8872b88e0c40" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 44389 } }
I20260619 02:40:04.158803 13447 ts_tablet_manager.cc:1428] T d9051b6a90864b74a6322f368390390e P f277ace414904d888dfcd10ba52bc2c5: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260619 02:40:04.158919 13285 heartbeater.cc:503] Master 127.5.241.126:45231 was elected leader, sending a full tablet report...
I20260619 02:40:04.158949 13446 ts_tablet_manager.cc:1428] T d9051b6a90864b74a6322f368390390e P 39f073a2df8741218340f6e1a84deb6a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:40:04.159049 13421 heartbeater.cc:503] Master 127.5.241.126:45231 was elected leader, sending a full tablet report...
I20260619 02:40:04.160917 13448 tablet_bootstrap.cc:492] T d9051b6a90864b74a6322f368390390e P e90938d5560646d08f1a8872b88e0c40: No bootstrap required, opened a new log
I20260619 02:40:04.161001 13448 ts_tablet_manager.cc:1397] T d9051b6a90864b74a6322f368390390e P e90938d5560646d08f1a8872b88e0c40: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260619 02:40:04.163241 13448 raft_consensus.cc:348] T d9051b6a90864b74a6322f368390390e P e90938d5560646d08f1a8872b88e0c40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f277ace414904d888dfcd10ba52bc2c5" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36493 } } peers { permanent_uuid: "39f073a2df8741218340f6e1a84deb6a" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 45923 } } peers { permanent_uuid: "e90938d5560646d08f1a8872b88e0c40" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 44389 } }
I20260619 02:40:04.163394 13448 raft_consensus.cc:374] T d9051b6a90864b74a6322f368390390e P e90938d5560646d08f1a8872b88e0c40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:04.163444 13448 raft_consensus.cc:729] T d9051b6a90864b74a6322f368390390e P e90938d5560646d08f1a8872b88e0c40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90938d5560646d08f1a8872b88e0c40, State: Initialized, Role: FOLLOWER
I20260619 02:40:04.163552 13448 consensus_queue.cc:260] T d9051b6a90864b74a6322f368390390e P e90938d5560646d08f1a8872b88e0c40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f277ace414904d888dfcd10ba52bc2c5" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36493 } } peers { permanent_uuid: "39f073a2df8741218340f6e1a84deb6a" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 45923 } } peers { permanent_uuid: "e90938d5560646d08f1a8872b88e0c40" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 44389 } }
I20260619 02:40:04.163801 13448 ts_tablet_manager.cc:1428] T d9051b6a90864b74a6322f368390390e P e90938d5560646d08f1a8872b88e0c40: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 02:40:04.163864 13148 heartbeater.cc:503] Master 127.5.241.126:45231 was elected leader, sending a full tablet report...
W20260619 02:40:04.233479 13287 tablet.cc:2369] T d9051b6a90864b74a6322f368390390e P f277ace414904d888dfcd10ba52bc2c5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:40:04.257400 13441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:04.254823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.124:36925 (local address 127.0.0.1:60400)
0619 02:40:04.254927 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:04.254940 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:04.254995 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:04.255244 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:04.255245 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:04.255246 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:40:04.255296 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:04.255297 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:04.255959 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:04.255960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:04.256685 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:04.256687 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:04.257218 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:04.257219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:04.257260 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:04.257263 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:40:04.257301 (+ 38us) client_negotiation.cc:240] Negotiation successful
0619 02:40:04.257338 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79}
I20260619 02:40:04.257391 13434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:04.254947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60400 (local address 127.5.241.124:36925)
0619 02:40:04.255016 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:04.255018 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:04.255104 (+ 86us) server_negotiation.cc:407] Connection header received
0619 02:40:04.255139 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:04.255140 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:04.255161 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:04.255224 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:40:04.255359 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:04.255880 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:04.256762 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:04.257147 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:04.257205 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:04.257261 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:04.257306 (+ 45us) server_negotiation.cc:299] Negotiation successful
0619 02:40:04.257332 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 02:40:04.257946 12978 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260619 02:40:04.258809 13007 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0305088fb8294ea485bdf6424e661096. Latest consensus state: current_term: 1 leader_uuid: "0305088fb8294ea485bdf6424e661096" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } } peers { permanent_uuid: "56c49503edc647499d6dcab7b82fcde9" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 38543 } attrs { promote: false } } peers { permanent_uuid: "1a6c09a7da984a169cf84a7187990d2b" member_type: NON_VOTER last_known_addr { host: "127.5.241.124" port: 36925 } attrs { promote: true } } }
I20260619 02:40:04.258924 13007 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [sys.catalog]: This master's current role is: LEARNER
I20260619 02:40:04.273243 13453 raft_consensus.cc:482] T d9051b6a90864b74a6322f368390390e P 39f073a2df8741218340f6e1a84deb6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:40:04.273386 13453 raft_consensus.cc:504] T d9051b6a90864b74a6322f368390390e P 39f073a2df8741218340f6e1a84deb6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f277ace414904d888dfcd10ba52bc2c5" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36493 } } peers { permanent_uuid: "39f073a2df8741218340f6e1a84deb6a" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 45923 } } peers { permanent_uuid: "e90938d5560646d08f1a8872b88e0c40" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 44389 } }
I20260619 02:40:04.273793 13453 leader_election.cc:288] T d9051b6a90864b74a6322f368390390e P 39f073a2df8741218340f6e1a84deb6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f277ace414904d888dfcd10ba52bc2c5 (127.5.241.66:36493), e90938d5560646d08f1a8872b88e0c40 (127.5.241.65:44389)
I20260619 02:40:04.276875 13444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:04.274114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:45831 (local address 127.5.241.66:36493)
0619 02:40:04.274187 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:04.274189 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:04.274202 (+ 13us) server_negotiation.cc:407] Connection header received
0619 02:40:04.274235 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:04.274236 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:04.274247 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:04.274323 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:40:04.274512 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:04.275175 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:04.276152 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:04.276519 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:04.276609 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:04.276618 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:04.276759 (+ 141us) server_negotiation.cc:299] Negotiation successful
0619 02:40:04.276803 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 02:40:04.276907 13426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:04.273991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.66:36493 (local address 127.5.241.67:45831)
0619 02:40:04.274047 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:04.274059 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:04.274103 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:04.274330 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:04.274331 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:04.274333 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:40:04.274413 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:04.274415 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:04.275254 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:04.275255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:04.276055 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:04.276057 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:04.276605 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:04.276606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:04.276686 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:04.276689 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:40:04.276730 (+ 41us) client_negotiation.cc:240] Negotiation successful
0619 02:40:04.276781 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":40}
I20260619 02:40:04.276907 13443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:04.274299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:42251 (local address 127.5.241.65:44389)
0619 02:40:04.274343 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:04.274345 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:04.274353 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:40:04.274378 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:04.274379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:04.274388 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:04.274451 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:40:04.274668 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:04.275205 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:04.276151 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:04.276516 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:04.276613 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:04.276621 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:04.276793 (+ 172us) server_negotiation.cc:299] Negotiation successful
0619 02:40:04.276833 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 02:40:04.276932 13425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:04.274122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:44389 (local address 127.5.241.67:42251)
0619 02:40:04.274212 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:04.274225 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:04.274286 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:04.274510 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:04.274511 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:04.274513 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:40:04.274570 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:04.274572 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:04.275308 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:04.275309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:04.276056 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:04.276058 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:04.276613 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:04.276614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:04.276694 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:04.276697 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:40:04.276778 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 02:40:04.276812 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260619 02:40:04.277325 13101 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9051b6a90864b74a6322f368390390e" candidate_uuid: "39f073a2df8741218340f6e1a84deb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90938d5560646d08f1a8872b88e0c40" is_pre_election: true
I20260619 02:40:04.277453 13101 raft_consensus.cc:2454] T d9051b6a90864b74a6322f368390390e P e90938d5560646d08f1a8872b88e0c40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39f073a2df8741218340f6e1a84deb6a in term 0.
I20260619 02:40:04.277457 13238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9051b6a90864b74a6322f368390390e" candidate_uuid: "39f073a2df8741218340f6e1a84deb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f277ace414904d888dfcd10ba52bc2c5" is_pre_election: true
I20260619 02:40:04.277606 13238 raft_consensus.cc:2454] T d9051b6a90864b74a6322f368390390e P f277ace414904d888dfcd10ba52bc2c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39f073a2df8741218340f6e1a84deb6a in term 0.
I20260619 02:40:04.277691 13310 leader_election.cc:302] T d9051b6a90864b74a6322f368390390e P 39f073a2df8741218340f6e1a84deb6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39f073a2df8741218340f6e1a84deb6a, e90938d5560646d08f1a8872b88e0c40; no voters:
I20260619 02:40:04.278134 13453 raft_consensus.cc:2781] T d9051b6a90864b74a6322f368390390e P 39f073a2df8741218340f6e1a84deb6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:40:04.278239 13453 raft_consensus.cc:482] T d9051b6a90864b74a6322f368390390e P 39f073a2df8741218340f6e1a84deb6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:40:04.278276 13453 raft_consensus.cc:3037] T d9051b6a90864b74a6322f368390390e P 39f073a2df8741218340f6e1a84deb6a [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:04.279166 13453 raft_consensus.cc:504] T d9051b6a90864b74a6322f368390390e P 39f073a2df8741218340f6e1a84deb6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f277ace414904d888dfcd10ba52bc2c5" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36493 } } peers { permanent_uuid: "39f073a2df8741218340f6e1a84deb6a" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 45923 } } peers { permanent_uuid: "e90938d5560646d08f1a8872b88e0c40" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 44389 } }
I20260619 02:40:04.279333 13453 leader_election.cc:288] T d9051b6a90864b74a6322f368390390e P 39f073a2df8741218340f6e1a84deb6a [CANDIDATE]: Term 1 election: Requested vote from peers f277ace414904d888dfcd10ba52bc2c5 (127.5.241.66:36493), e90938d5560646d08f1a8872b88e0c40 (127.5.241.65:44389)
I20260619 02:40:04.279564 13238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9051b6a90864b74a6322f368390390e" candidate_uuid: "39f073a2df8741218340f6e1a84deb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f277ace414904d888dfcd10ba52bc2c5"
I20260619 02:40:04.279639 13238 raft_consensus.cc:3037] T d9051b6a90864b74a6322f368390390e P f277ace414904d888dfcd10ba52bc2c5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:04.279587 13101 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9051b6a90864b74a6322f368390390e" candidate_uuid: "39f073a2df8741218340f6e1a84deb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90938d5560646d08f1a8872b88e0c40"
I20260619 02:40:04.279711 13101 raft_consensus.cc:3037] T d9051b6a90864b74a6322f368390390e P e90938d5560646d08f1a8872b88e0c40 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:04.280403 13238 raft_consensus.cc:2454] T d9051b6a90864b74a6322f368390390e P f277ace414904d888dfcd10ba52bc2c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39f073a2df8741218340f6e1a84deb6a in term 1.
I20260619 02:40:04.280501 13101 raft_consensus.cc:2454] T d9051b6a90864b74a6322f368390390e P e90938d5560646d08f1a8872b88e0c40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39f073a2df8741218340f6e1a84deb6a in term 1.
I20260619 02:40:04.280586 13310 leader_election.cc:302] T d9051b6a90864b74a6322f368390390e P 39f073a2df8741218340f6e1a84deb6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39f073a2df8741218340f6e1a84deb6a, f277ace414904d888dfcd10ba52bc2c5; no voters:
I20260619 02:40:04.280685 13453 raft_consensus.cc:2781] T d9051b6a90864b74a6322f368390390e P 39f073a2df8741218340f6e1a84deb6a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:40:04.280881 13453 raft_consensus.cc:686] T d9051b6a90864b74a6322f368390390e P 39f073a2df8741218340f6e1a84deb6a [term 1 LEADER]: Becoming Leader. State: Replica: 39f073a2df8741218340f6e1a84deb6a, State: Running, Role: LEADER
I20260619 02:40:04.280985 13453 consensus_queue.cc:237] T d9051b6a90864b74a6322f368390390e P 39f073a2df8741218340f6e1a84deb6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f277ace414904d888dfcd10ba52bc2c5" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36493 } } peers { permanent_uuid: "39f073a2df8741218340f6e1a84deb6a" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 45923 } } peers { permanent_uuid: "e90938d5560646d08f1a8872b88e0c40" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 44389 } }
I20260619 02:40:04.281862 12184 catalog_manager.cc:5495] T d9051b6a90864b74a6322f368390390e P 39f073a2df8741218340f6e1a84deb6a reported cstate change: term changed from 0 to 1, leader changed from <none> to 39f073a2df8741218340f6e1a84deb6a (127.5.241.67). New cstate: current_term: 1 leader_uuid: "39f073a2df8741218340f6e1a84deb6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f277ace414904d888dfcd10ba52bc2c5" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39f073a2df8741218340f6e1a84deb6a" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 45923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e90938d5560646d08f1a8872b88e0c40" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 44389 } health_report { overall_health: UNKNOWN } } }
I20260619 02:40:04.282505 12744 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0305088fb8294ea485bdf6424e661096" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } } peers { permanent_uuid: "56c49503edc647499d6dcab7b82fcde9" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 38543 } attrs { promote: false } } peers { permanent_uuid: "1a6c09a7da984a169cf84a7187990d2b" member_type: NON_VOTER last_known_addr { host: "127.5.241.124" port: 36925 } attrs { promote: true } } }
I20260619 02:40:04.282599 12744 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096 [sys.catalog]: This master's current role is: LEADER
I20260619 02:40:04.283349 13460 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096: attempting to promote NON_VOTER 1a6c09a7da984a169cf84a7187990d2b to VOTER
I20260619 02:40:04.283486 13460 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } } peers { permanent_uuid: "56c49503edc647499d6dcab7b82fcde9" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 38543 } attrs { promote: false } } peers { permanent_uuid: "1a6c09a7da984a169cf84a7187990d2b" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 36925 } attrs { promote: false } }
I20260619 02:40:04.284216 12978 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [term 1 LEARNER]: Refusing update from remote peer 0305088fb8294ea485bdf6424e661096: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260619 02:40:04.284389 12792 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [term 1 FOLLOWER]: Refusing update from remote peer 0305088fb8294ea485bdf6424e661096: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260619 02:40:04.284515 13461 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a6c09a7da984a169cf84a7187990d2b" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 36925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260619 02:40:04.284614 12932 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56c49503edc647499d6dcab7b82fcde9" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 38543 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260619 02:40:04.285220 12744 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 1a6c09a7da984a169cf84a7187990d2b (127.5.241.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } } peers { permanent_uuid: "56c49503edc647499d6dcab7b82fcde9" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 38543 } attrs { promote: false } } peers { permanent_uuid: "1a6c09a7da984a169cf84a7187990d2b" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 36925 } attrs { promote: false } } }
I20260619 02:40:04.285346 12792 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 1a6c09a7da984a169cf84a7187990d2b (127.5.241.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } } peers { permanent_uuid: "56c49503edc647499d6dcab7b82fcde9" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 38543 } attrs { promote: false } } peers { permanent_uuid: "1a6c09a7da984a169cf84a7187990d2b" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 36925 } attrs { promote: false } } }
I20260619 02:40:04.286180 12744 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0305088fb8294ea485bdf6424e661096" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } } peers { permanent_uuid: "56c49503edc647499d6dcab7b82fcde9" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 38543 } attrs { promote: false } } peers { permanent_uuid: "1a6c09a7da984a169cf84a7187990d2b" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 36925 } attrs { promote: false } } }
I20260619 02:40:04.286224 13461 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0305088fb8294ea485bdf6424e661096" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } } peers { permanent_uuid: "56c49503edc647499d6dcab7b82fcde9" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 38543 } attrs { promote: false } } peers { permanent_uuid: "1a6c09a7da984a169cf84a7187990d2b" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 36925 } attrs { promote: false } } }
I20260619 02:40:04.286379 13461 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096 [sys.catalog]: This master's current role is: LEADER
I20260619 02:40:04.286268 12932 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0305088fb8294ea485bdf6424e661096" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } } peers { permanent_uuid: "56c49503edc647499d6dcab7b82fcde9" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 38543 } attrs { promote: false } } peers { permanent_uuid: "1a6c09a7da984a169cf84a7187990d2b" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 36925 } attrs { promote: false } } }
I20260619 02:40:04.286448 12932 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096 [sys.catalog]: This master's current role is: LEADER
I20260619 02:40:04.286485 13463 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0305088fb8294ea485bdf6424e661096" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } } peers { permanent_uuid: "56c49503edc647499d6dcab7b82fcde9" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 38543 } attrs { promote: false } } peers { permanent_uuid: "1a6c09a7da984a169cf84a7187990d2b" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 36925 } attrs { promote: false } } }
I20260619 02:40:04.286561 13463 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:40:04.286494 12978 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 1a6c09a7da984a169cf84a7187990d2b (127.5.241.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } } peers { permanent_uuid: "56c49503edc647499d6dcab7b82fcde9" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 38543 } attrs { promote: false } } peers { permanent_uuid: "1a6c09a7da984a169cf84a7187990d2b" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 36925 } attrs { promote: false } } }
I20260619 02:40:04.287377 13007 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0305088fb8294ea485bdf6424e661096" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } } peers { permanent_uuid: "56c49503edc647499d6dcab7b82fcde9" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 38543 } attrs { promote: false } } peers { permanent_uuid: "1a6c09a7da984a169cf84a7187990d2b" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 36925 } attrs { promote: false } } }
I20260619 02:40:04.287465 13007 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:40:04.286273 12744 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096 [sys.catalog]: This master's current role is: LEADER
I20260619 02:40:04.293645 13445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:04.291069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39130 (local address 127.5.241.67:45923)
0619 02:40:04.291134 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:04.291136 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:04.291145 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:40:04.291184 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:04.291184 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:04.291203 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:04.291301 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:40:04.291453 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:04.292063 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:04.292849 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:04.293057 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:04.293133 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:04.293313 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:40:04.293423 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:40:04.293518 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:04.293543 (+ 25us) server_negotiation.cc:299] Negotiation successful
0619 02:40:04.293562 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 02:40:04.295580 13238 raft_consensus.cc:1261] T d9051b6a90864b74a6322f368390390e P f277ace414904d888dfcd10ba52bc2c5 [term 1 FOLLOWER]: Refusing update from remote peer 39f073a2df8741218340f6e1a84deb6a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:40:04.295580 13101 raft_consensus.cc:1261] T d9051b6a90864b74a6322f368390390e P e90938d5560646d08f1a8872b88e0c40 [term 1 FOLLOWER]: Refusing update from remote peer 39f073a2df8741218340f6e1a84deb6a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:40:04.295948 13458 consensus_queue.cc:1035] T d9051b6a90864b74a6322f368390390e P 39f073a2df8741218340f6e1a84deb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f277ace414904d888dfcd10ba52bc2c5" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:40:04.296087 13453 consensus_queue.cc:1035] T d9051b6a90864b74a6322f368390390e P 39f073a2df8741218340f6e1a84deb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e90938d5560646d08f1a8872b88e0c40" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 44389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:40:04.300210 13467 mvcc.cc:205] Tried to move back new op lower bound from 7298403550391775232 to 7298403550335623168. Current Snapshot: MvccSnapshot[applied={T|T < 7298403550391775232 or (T in {7298403550391775232})}]
I20260619 02:40:04.301697 12184 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51048:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:40:04.301786 12184 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51048:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:40:04.301784 13466 mvcc.cc:205] Tried to move back new op lower bound from 7298403550391775232 to 7298403550335623168. Current Snapshot: MvccSnapshot[applied={T|T < 7298403550391775232}]
I20260619 02:40:04.303256 12184 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0305088fb8294ea485bdf6424e661096: Sending DeleteTablet for 3 replicas of tablet d9051b6a90864b74a6322f368390390e
I20260619 02:40:04.303577 13354 tablet_service.cc:1514] Processing DeleteTablet for tablet d9051b6a90864b74a6322f368390390e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:40:04 UTC) from {username='oryx', principal='oryx/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:39114
I20260619 02:40:04.304131 13081 tablet_service.cc:1514] Processing DeleteTablet for tablet d9051b6a90864b74a6322f368390390e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:40:04 UTC) from {username='oryx', principal='oryx/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:39874
I20260619 02:40:04.304352 13476 tablet_replica.cc:330] T d9051b6a90864b74a6322f368390390e P e90938d5560646d08f1a8872b88e0c40: stopping tablet replica
I20260619 02:40:04.304474 13476 raft_consensus.cc:2229] T d9051b6a90864b74a6322f368390390e P e90938d5560646d08f1a8872b88e0c40 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:40:04.304463 13218 tablet_service.cc:1514] Processing DeleteTablet for tablet d9051b6a90864b74a6322f368390390e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:40:04 UTC) from {username='oryx', principal='oryx/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:59488
I20260619 02:40:04.304579 13476 raft_consensus.cc:2258] T d9051b6a90864b74a6322f368390390e P e90938d5560646d08f1a8872b88e0c40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:40:04.304623 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 12153
I20260619 02:40:04.305194 13476 ts_tablet_manager.cc:1905] T d9051b6a90864b74a6322f368390390e P e90938d5560646d08f1a8872b88e0c40: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:40:04.306874 13476 ts_tablet_manager.cc:1918] T d9051b6a90864b74a6322f368390390e P e90938d5560646d08f1a8872b88e0c40: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:40:04.306958 13476 log.cc:1196] T d9051b6a90864b74a6322f368390390e P e90938d5560646d08f1a8872b88e0c40: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-0/wal/wals/d9051b6a90864b74a6322f368390390e
I20260619 02:40:04.307226 13476 ts_tablet_manager.cc:1939] T d9051b6a90864b74a6322f368390390e P e90938d5560646d08f1a8872b88e0c40: Deleting consensus metadata
W20260619 02:40:04.311005 13034 connection.cc:535] server connection from 127.0.0.1:39874 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 02:40:04.312052 13434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:04.303944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60408 (local address 127.5.241.124:36925)
0619 02:40:04.304002 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:04.304004 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:04.304115 (+ 111us) server_negotiation.cc:407] Connection header received
0619 02:40:04.304222 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:04.304223 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:04.304235 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:04.304269 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:04.305141 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:04.305831 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:04.306981 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:04.307302 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:04.307350 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:04.309098 (+ 1748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:04.309099 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:04.309100 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:04.309108 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:04.310580 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:04.311144 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:04.311145 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:04.311145 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:04.311207 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:04.311419 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:04.311420 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:04.311420 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:04.311555 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:04.311630 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:04.311852 (+ 222us) server_negotiation.cc:299] Negotiation successful
0619 02:40:04.311957 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 02:40:04.314919 13474 tablet_replica.cc:330] T d9051b6a90864b74a6322f368390390e P 39f073a2df8741218340f6e1a84deb6a: stopping tablet replica
I20260619 02:40:04.315114 13477 tablet_replica.cc:330] T d9051b6a90864b74a6322f368390390e P f277ace414904d888dfcd10ba52bc2c5: stopping tablet replica
I20260619 02:40:04.315306 13477 raft_consensus.cc:2229] T d9051b6a90864b74a6322f368390390e P f277ace414904d888dfcd10ba52bc2c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:40:04.315447 13474 raft_consensus.cc:2229] T d9051b6a90864b74a6322f368390390e P 39f073a2df8741218340f6e1a84deb6a [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:40:04.315613 13474 raft_consensus.cc:2258] T d9051b6a90864b74a6322f368390390e P 39f073a2df8741218340f6e1a84deb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:40:04.315765 13477 raft_consensus.cc:2258] T d9051b6a90864b74a6322f368390390e P f277ace414904d888dfcd10ba52bc2c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:40:04.317087 13474 ts_tablet_manager.cc:1905] T d9051b6a90864b74a6322f368390390e P 39f073a2df8741218340f6e1a84deb6a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:40:04.317322 13477 ts_tablet_manager.cc:1905] T d9051b6a90864b74a6322f368390390e P f277ace414904d888dfcd10ba52bc2c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:40:04.319796 13474 ts_tablet_manager.cc:1918] T d9051b6a90864b74a6322f368390390e P 39f073a2df8741218340f6e1a84deb6a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:40:04.319877 13474 log.cc:1196] T d9051b6a90864b74a6322f368390390e P 39f073a2df8741218340f6e1a84deb6a: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-2/wal/wals/d9051b6a90864b74a6322f368390390e
I20260619 02:40:04.320216 13474 ts_tablet_manager.cc:1939] T d9051b6a90864b74a6322f368390390e P 39f073a2df8741218340f6e1a84deb6a: Deleting consensus metadata
I20260619 02:40:04.320842 13434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:04.312729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60420 (local address 127.5.241.124:36925)
0619 02:40:04.312766 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:04.312768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:04.313021 (+ 253us) server_negotiation.cc:407] Connection header received
0619 02:40:04.313118 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:04.313119 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:04.313131 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:04.313160 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:04.313877 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:04.314497 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:04.315575 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:04.315869 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:04.315926 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:04.317806 (+ 1880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:04.317807 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:04.317808 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:04.317815 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:04.319567 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:04.320000 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:04.320001 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:04.320001 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:04.320058 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:04.320259 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:04.320259 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:04.320260 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:04.320383 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:04.320456 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:04.320669 (+ 213us) server_negotiation.cc:299] Negotiation successful
0619 02:40:04.320746 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260619 02:40:04.321866 13308 connection.cc:441] server connection from 127.0.0.1:39114 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39114 (request call id 1) could send its response
I20260619 02:40:04.322168 13477 ts_tablet_manager.cc:1918] T d9051b6a90864b74a6322f368390390e P f277ace414904d888dfcd10ba52bc2c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:40:04.322252 13477 log.cc:1196] T d9051b6a90864b74a6322f368390390e P f277ace414904d888dfcd10ba52bc2c5: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-1/wal/wals/d9051b6a90864b74a6322f368390390e
I20260619 02:40:04.322546 13477 ts_tablet_manager.cc:1939] T d9051b6a90864b74a6322f368390390e P f277ace414904d888dfcd10ba52bc2c5: Deleting consensus metadata
W20260619 02:40:04.322948 13171 connection.cc:441] server connection from 127.0.0.1:59488 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:59488 (request call id 1) could send its response
I20260619 02:40:04.323076 12821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:04.312798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37480 (local address 127.5.241.125:38543)
0619 02:40:04.312838 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:04.312840 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:04.313269 (+ 429us) server_negotiation.cc:407] Connection header received
0619 02:40:04.313322 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:04.313323 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:04.313336 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:04.313396 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:04.314237 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:04.314882 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:04.315839 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:04.316085 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:04.316121 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:04.319710 (+ 3589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:04.319711 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:04.319712 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:04.319721 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:04.321848 (+ 2127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:04.322244 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:04.322245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:04.322245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:04.322304 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:04.322506 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:04.322507 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:04.322508 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:04.322664 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:04.322764 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:04.322893 (+ 129us) server_negotiation.cc:299] Negotiation successful
0619 02:40:04.322980 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 02:40:05.301537 13485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:05.301206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45231 (local address 127.5.241.67:37179)
0619 02:40:05.301385 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:05.301476 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":122,"threads_started":1}
I20260619 02:40:05.301865 13486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:05.301513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45231 (local address 127.5.241.66:48973)
0619 02:40:05.301741 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:05.301811 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":46,"threads_started":1}
W20260619 02:40:05.302062 13285 heartbeater.cc:644] Failed to heartbeat to 127.5.241.126:45231 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.5.241.126:45231: connect: Connection refused (error 111)
W20260619 02:40:05.302373 13421 heartbeater.cc:644] Failed to heartbeat to 127.5.241.126:45231 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.5.241.126:45231: connect: Connection refused (error 111)
I20260619 02:40:05.304177 13487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:05.303986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45231 (local address 127.5.241.65:45801)
0619 02:40:05.304095 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:05.304143 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":60,"threads_started":1}
W20260619 02:40:05.304405 13148 heartbeater.cc:644] Failed to heartbeat to 127.5.241.126:45231 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.5.241.126:45231: connect: Connection refused (error 111)
I20260619 02:40:05.803772 13489 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0305088fb8294ea485bdf6424e661096)
I20260619 02:40:05.803771 13488 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0305088fb8294ea485bdf6424e661096)
I20260619 02:40:05.803885 13489 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } } peers { permanent_uuid: "56c49503edc647499d6dcab7b82fcde9" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 38543 } attrs { promote: false } } peers { permanent_uuid: "1a6c09a7da984a169cf84a7187990d2b" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 36925 } attrs { promote: false } }
I20260619 02:40:05.803934 13488 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } } peers { permanent_uuid: "56c49503edc647499d6dcab7b82fcde9" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 38543 } attrs { promote: false } } peers { permanent_uuid: "1a6c09a7da984a169cf84a7187990d2b" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 36925 } attrs { promote: false } }
I20260619 02:40:05.804221 13488 leader_election.cc:288] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0305088fb8294ea485bdf6424e661096 (127.5.241.126:45231), 1a6c09a7da984a169cf84a7187990d2b (127.5.241.124:36925)
I20260619 02:40:05.804374 13489 leader_election.cc:288] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0305088fb8294ea485bdf6424e661096 (127.5.241.126:45231), 56c49503edc647499d6dcab7b82fcde9 (127.5.241.125:38543)
I20260619 02:40:05.804571 13490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:05.804234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45231 (local address 127.0.0.1:51848)
0619 02:40:05.804371 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:05.804522 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":48,"threads_started":1}
I20260619 02:40:05.804647 13493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:05.804331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45231 (local address 127.0.0.1:51854)
0619 02:40:05.804542 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:05.804601 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":49,"threads_started":1}
W20260619 02:40:05.804770 12767 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45231: connect: Connection refused (error 111)
W20260619 02:40:05.804845 12953 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45231: connect: Connection refused (error 111)
I20260619 02:40:05.805167 13493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:05.805086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45231 (local address 127.0.0.1:51856)
0619 02:40:05.805120 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:05.805144 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260619 02:40:05.805244 12767 leader_election.cc:334] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0305088fb8294ea485bdf6424e661096 (127.5.241.126:45231): Network error: Client connection negotiation failed: client connection to 127.5.241.126:45231: connect: Connection refused (error 111)
I20260619 02:40:05.805410 13490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:05.805274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45231 (local address 127.0.0.1:51862)
0619 02:40:05.805358 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:05.805384 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260619 02:40:05.805476 12953 leader_election.cc:334] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0305088fb8294ea485bdf6424e661096 (127.5.241.126:45231): Network error: Client connection negotiation failed: client connection to 127.5.241.126:45231: connect: Connection refused (error 111)
I20260619 02:40:05.807503 13495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:05.804698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55182 (local address 127.5.241.124:36925)
0619 02:40:05.804816 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:05.804819 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:05.804828 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:40:05.804853 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:05.804854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:05.804864 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:05.804940 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:40:05.805143 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:05.805807 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:05.806720 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:05.807142 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:05.807234 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:05.807245 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:05.807380 (+ 135us) server_negotiation.cc:299] Negotiation successful
0619 02:40:05.807428 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":101,"thread_start_us":64,"threads_started":1}
I20260619 02:40:05.807560 13491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:05.804259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.125:38543 (local address 127.0.0.1:59330)
0619 02:40:05.804493 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:05.804506 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:05.804565 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:05.804962 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:05.804963 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:05.804966 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:40:05.805027 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:05.805030 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:05.805889 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:05.805890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:05.806629 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:05.806631 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:05.807237 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:05.807238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:05.807303 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:05.807306 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:40:05.807381 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 02:40:05.807437 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":5,"thread_start_us":87,"threads_started":1}
I20260619 02:40:05.807529 13494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:05.804448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59330 (local address 127.5.241.125:38543)
0619 02:40:05.804805 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:05.804806 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:05.804813 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:40:05.804835 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:05.804836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:05.804845 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:05.805092 (+ 247us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:40:05.805127 (+ 35us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:05.805800 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:05.806735 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:05.807140 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:05.807234 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:05.807245 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:05.807388 (+ 143us) server_negotiation.cc:299] Negotiation successful
0619 02:40:05.807432 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":42,"threads_started":1}
I20260619 02:40:05.807528 13492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:05.804438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.124:36925 (local address 127.0.0.1:55182)
0619 02:40:05.804666 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:05.804676 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:05.804722 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:05.805008 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:05.805009 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:05.805010 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:40:05.805063 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:05.805065 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:05.805889 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:05.805891 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:05.806629 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:05.806631 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:05.807236 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:05.807237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:05.807297 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:05.807300 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:40:05.807373 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 02:40:05.807419 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":53,"threads_started":1}
I20260619 02:40:05.807801 12792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a6c09a7da984a169cf84a7187990d2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "56c49503edc647499d6dcab7b82fcde9" is_pre_election: true
I20260619 02:40:05.807906 12792 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a6c09a7da984a169cf84a7187990d2b in term 1.
I20260619 02:40:05.808032 12955 leader_election.cc:302] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1a6c09a7da984a169cf84a7187990d2b, 56c49503edc647499d6dcab7b82fcde9; no voters: 0305088fb8294ea485bdf6424e661096
I20260619 02:40:05.807994 12978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "56c49503edc647499d6dcab7b82fcde9" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "1a6c09a7da984a169cf84a7187990d2b" is_pre_election: true
I20260619 02:40:05.808113 12978 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56c49503edc647499d6dcab7b82fcde9 in term 1.
I20260619 02:40:05.808159 13489 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 02:40:05.808207 13489 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0305088fb8294ea485bdf6424e661096)
I20260619 02:40:05.808233 13489 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [term 1 FOLLOWER]: Advancing to term 2
I20260619 02:40:05.808260 12766 leader_election.cc:302] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1a6c09a7da984a169cf84a7187990d2b, 56c49503edc647499d6dcab7b82fcde9; no voters: 0305088fb8294ea485bdf6424e661096
I20260619 02:40:05.808455 13488 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 02:40:05.808522 13488 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0305088fb8294ea485bdf6424e661096)
I20260619 02:40:05.808563 13488 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [term 1 FOLLOWER]: Advancing to term 2
I20260619 02:40:05.809047 13489 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } } peers { permanent_uuid: "56c49503edc647499d6dcab7b82fcde9" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 38543 } attrs { promote: false } } peers { permanent_uuid: "1a6c09a7da984a169cf84a7187990d2b" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 36925 } attrs { promote: false } }
I20260619 02:40:05.809223 13489 leader_election.cc:288] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [CANDIDATE]: Term 2 election: Requested vote from peers 0305088fb8294ea485bdf6424e661096 (127.5.241.126:45231), 56c49503edc647499d6dcab7b82fcde9 (127.5.241.125:38543)
I20260619 02:40:05.809379 13491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:05.809293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45231 (local address 127.0.0.1:51868)
0619 02:40:05.809324 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:05.809352 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260619 02:40:05.809393 13488 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } } peers { permanent_uuid: "56c49503edc647499d6dcab7b82fcde9" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 38543 } attrs { promote: false } } peers { permanent_uuid: "1a6c09a7da984a169cf84a7187990d2b" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 36925 } attrs { promote: false } }
I20260619 02:40:05.809554 13488 leader_election.cc:288] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [CANDIDATE]: Term 2 election: Requested vote from peers 0305088fb8294ea485bdf6424e661096 (127.5.241.126:45231), 1a6c09a7da984a169cf84a7187990d2b (127.5.241.124:36925)
I20260619 02:40:05.809679 12978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "56c49503edc647499d6dcab7b82fcde9" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "1a6c09a7da984a169cf84a7187990d2b"
I20260619 02:40:05.809736 12792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a6c09a7da984a169cf84a7187990d2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "56c49503edc647499d6dcab7b82fcde9"
I20260619 02:40:05.809790 12978 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 56c49503edc647499d6dcab7b82fcde9 in current term 2: Already voted for candidate 1a6c09a7da984a169cf84a7187990d2b in this term.
I20260619 02:40:05.809826 12792 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 1a6c09a7da984a169cf84a7187990d2b in current term 2: Already voted for candidate 56c49503edc647499d6dcab7b82fcde9 in this term.
I20260619 02:40:05.809803 13492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:05.809616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45231 (local address 127.0.0.1:51880)
0619 02:40:05.809699 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:05.809726 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260619 02:40:05.809994 13491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:05.809609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45231 (local address 127.0.0.1:51878)
0619 02:40:05.809645 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:05.809667 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260619 02:40:05.810086 12953 leader_election.cc:334] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 0305088fb8294ea485bdf6424e661096 (127.5.241.126:45231): Network error: Client connection negotiation failed: client connection to 127.5.241.126:45231: connect: Connection refused (error 111)
I20260619 02:40:05.810113 13492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:05.810037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45231 (local address 127.0.0.1:51888)
0619 02:40:05.810073 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:05.810093 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260619 02:40:05.810132 12953 leader_election.cc:302] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1a6c09a7da984a169cf84a7187990d2b; no voters: 0305088fb8294ea485bdf6424e661096, 56c49503edc647499d6dcab7b82fcde9
W20260619 02:40:05.810189 12767 leader_election.cc:334] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 0305088fb8294ea485bdf6424e661096 (127.5.241.126:45231): Network error: Client connection negotiation failed: client connection to 127.5.241.126:45231: connect: Connection refused (error 111)
I20260619 02:40:05.810207 13489 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 02:40:05.810221 12767 leader_election.cc:302] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56c49503edc647499d6dcab7b82fcde9; no voters: 0305088fb8294ea485bdf6424e661096, 1a6c09a7da984a169cf84a7187990d2b
I20260619 02:40:05.810302 13488 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 02:40:06.302860 13498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:06.302402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45231 (local address 127.5.241.66:38883)
0619 02:40:06.302744 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:06.302825 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":69,"threads_started":1}
I20260619 02:40:06.302857 13499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:06.302643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45231 (local address 127.5.241.67:53791)
0619 02:40:06.302743 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:06.302802 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82,"thread_start_us":47,"threads_started":1}
I20260619 02:40:06.304852 13500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:06.304676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45231 (local address 127.5.241.65:45047)
0619 02:40:06.304782 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:06.304818 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":64,"threads_started":1}
I20260619 02:40:07.303654 13501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:07.303308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45231 (local address 127.5.241.67:44451)
0619 02:40:07.303550 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:07.303593 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":65,"threads_started":1}
I20260619 02:40:07.303783 13502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:07.303486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45231 (local address 127.5.241.66:40659)
0619 02:40:07.303709 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:07.303742 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":50,"threads_started":1}
I20260619 02:40:07.305509 13503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:07.305247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45231 (local address 127.5.241.65:40067)
0619 02:40:07.305420 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:07.305473 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":100,"threads_started":1}
I20260619 02:40:07.628137 13504 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:40:07.628257 13504 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } } peers { permanent_uuid: "56c49503edc647499d6dcab7b82fcde9" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 38543 } attrs { promote: false } } peers { permanent_uuid: "1a6c09a7da984a169cf84a7187990d2b" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 36925 } attrs { promote: false } }
I20260619 02:40:07.628440 13504 leader_election.cc:288] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0305088fb8294ea485bdf6424e661096 (127.5.241.126:45231), 56c49503edc647499d6dcab7b82fcde9 (127.5.241.125:38543)
I20260619 02:40:07.628824 13505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:07.628577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45231 (local address 127.0.0.1:52120)
0619 02:40:07.628714 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:07.628769 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":66,"threads_started":1}
I20260619 02:40:07.628875 12792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a6c09a7da984a169cf84a7187990d2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "56c49503edc647499d6dcab7b82fcde9" is_pre_election: true
I20260619 02:40:07.628975 12792 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a6c09a7da984a169cf84a7187990d2b in term 2.
I20260619 02:40:07.629159 12955 leader_election.cc:302] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a6c09a7da984a169cf84a7187990d2b, 56c49503edc647499d6dcab7b82fcde9; no voters:
I20260619 02:40:07.629343 13504 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 02:40:07.629403 13504 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:40:07.629457 13505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:07.629301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45231 (local address 127.0.0.1:52134)
0619 02:40:07.629397 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:07.629431 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260619 02:40:07.629458 13504 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [term 2 FOLLOWER]: Advancing to term 3
W20260619 02:40:07.629565 12953 leader_election.cc:334] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0305088fb8294ea485bdf6424e661096 (127.5.241.126:45231): Network error: Client connection negotiation failed: client connection to 127.5.241.126:45231: connect: Connection refused (error 111)
I20260619 02:40:07.630285 13504 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } } peers { permanent_uuid: "56c49503edc647499d6dcab7b82fcde9" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 38543 } attrs { promote: false } } peers { permanent_uuid: "1a6c09a7da984a169cf84a7187990d2b" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 36925 } attrs { promote: false } }
I20260619 02:40:07.630506 13504 leader_election.cc:288] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [CANDIDATE]: Term 3 election: Requested vote from peers 0305088fb8294ea485bdf6424e661096 (127.5.241.126:45231), 56c49503edc647499d6dcab7b82fcde9 (127.5.241.125:38543)
I20260619 02:40:07.630616 13505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:07.630506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45231 (local address 127.0.0.1:52140)
0619 02:40:07.630553 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:07.630587 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260619 02:40:07.630589 12792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a6c09a7da984a169cf84a7187990d2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "56c49503edc647499d6dcab7b82fcde9"
I20260619 02:40:07.630661 12792 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [term 2 FOLLOWER]: Advancing to term 3
I20260619 02:40:07.631501 12792 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a6c09a7da984a169cf84a7187990d2b in term 3.
I20260619 02:40:07.631668 12955 leader_election.cc:302] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a6c09a7da984a169cf84a7187990d2b, 56c49503edc647499d6dcab7b82fcde9; no voters:
I20260619 02:40:07.631763 13504 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [term 3 FOLLOWER]: Leader election won for term 3
I20260619 02:40:07.631877 13504 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [term 3 LEADER]: Becoming Leader. State: Replica: 1a6c09a7da984a169cf84a7187990d2b, State: Running, Role: LEADER
I20260619 02:40:07.631964 13504 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } } peers { permanent_uuid: "56c49503edc647499d6dcab7b82fcde9" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 38543 } attrs { promote: false } } peers { permanent_uuid: "1a6c09a7da984a169cf84a7187990d2b" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 36925 } attrs { promote: false } }
I20260619 02:40:07.632069 13505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:07.631950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45231 (local address 127.0.0.1:52146)
0619 02:40:07.631996 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:07.632042 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260619 02:40:07.632123 12953 leader_election.cc:334] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 0305088fb8294ea485bdf6424e661096 (127.5.241.126:45231): Network error: Client connection negotiation failed: client connection to 127.5.241.126:45231: connect: Connection refused (error 111)
I20260619 02:40:07.632298 13507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a6c09a7da984a169cf84a7187990d2b. Latest consensus state: current_term: 3 leader_uuid: "1a6c09a7da984a169cf84a7187990d2b" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } } peers { permanent_uuid: "56c49503edc647499d6dcab7b82fcde9" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 38543 } attrs { promote: false } } peers { permanent_uuid: "1a6c09a7da984a169cf84a7187990d2b" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 36925 } attrs { promote: false } } }
I20260619 02:40:07.632508 13507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [sys.catalog]: This master's current role is: LEADER
I20260619 02:40:07.632707 13509 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:40:07.633162 13509 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:40:07.633257 13509 catalog_manager.cc:1259] Loaded cluster ID: c32bc2d7e2e547d6adb2678ff8bbe53c
I20260619 02:40:07.633291 13509 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:40:07.633536 13509 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:40:07.633627 13509 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b: Loaded TSK: 0
I20260619 02:40:07.633785 13509 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:40:07.637811 12966 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60420:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:40:07.639524 12966 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:40:07.640537 13505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:07.640421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45231 (local address 127.0.0.1:52156)
0619 02:40:07.640481 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:07.640513 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260619 02:40:07.640656 12792 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [term 3 FOLLOWER]: Refusing update from remote peer 1a6c09a7da984a169cf84a7187990d2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260619 02:40:07.640877 13504 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "56c49503edc647499d6dcab7b82fcde9" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 38543 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260619 02:40:07.640938 13505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:07.640805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45231 (local address 127.0.0.1:52172)
0619 02:40:07.640850 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:07.640894 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260619 02:40:07.641048 12953 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b -> Peer 0305088fb8294ea485bdf6424e661096 (127.5.241.126:45231): Couldn't send request to peer 0305088fb8294ea485bdf6424e661096. Status: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45231: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260619 02:40:07.641525 13511 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a6c09a7da984a169cf84a7187990d2b. Latest consensus state: current_term: 3 leader_uuid: "1a6c09a7da984a169cf84a7187990d2b" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } } peers { permanent_uuid: "56c49503edc647499d6dcab7b82fcde9" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 38543 } attrs { promote: false } } peers { permanent_uuid: "1a6c09a7da984a169cf84a7187990d2b" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 36925 } attrs { promote: false } } }
I20260619 02:40:07.641635 13511 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:40:07.642035 13504 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "1a6c09a7da984a169cf84a7187990d2b" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } } peers { permanent_uuid: "56c49503edc647499d6dcab7b82fcde9" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 38543 } attrs { promote: false } } peers { permanent_uuid: "1a6c09a7da984a169cf84a7187990d2b" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 36925 } attrs { promote: false } } }
I20260619 02:40:07.642127 13504 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b [sys.catalog]: This master's current role is: LEADER
I20260619 02:40:07.642431 13511 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "1a6c09a7da984a169cf84a7187990d2b" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "0305088fb8294ea485bdf6424e661096" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45231 } } peers { permanent_uuid: "56c49503edc647499d6dcab7b82fcde9" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 38543 } attrs { promote: false } } peers { permanent_uuid: "1a6c09a7da984a169cf84a7187990d2b" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 36925 } attrs { promote: false } } }
I20260619 02:40:07.642503 13511 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56c49503edc647499d6dcab7b82fcde9 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:40:07.647282 13505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:07.643659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.66:36493 (local address 127.0.0.1:56284)
0619 02:40:07.643702 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:07.643713 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:07.643761 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:07.644185 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:07.644186 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:07.644187 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:40:07.644240 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:07.644242 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:07.645125 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:07.645126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:07.646378 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:07.646379 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:07.647025 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:07.647026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:07.647092 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:07.647096 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:40:07.647146 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 02:40:07.647198 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29,"mutex_wait_us":53}
I20260619 02:40:07.647365 13519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:07.643989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33974 (local address 127.5.241.65:44389)
0619 02:40:07.644154 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:07.644156 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:07.644164 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:40:07.644187 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:07.644188 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:07.644198 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:07.644262 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:40:07.644394 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:07.645183 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:07.646456 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:07.647044 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:07.647113 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:07.647126 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:07.647259 (+ 133us) server_negotiation.cc:299] Negotiation successful
0619 02:40:07.647298 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":66,"threads_started":1}
I20260619 02:40:07.647390 13520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:07.644107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51814 (local address 127.5.241.67:45923)
0619 02:40:07.644343 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:07.644345 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:07.644353 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:40:07.644377 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:07.644378 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:07.644389 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:07.644460 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:40:07.644585 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:07.645277 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:07.646323 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:07.646930 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:07.647251 (+ 321us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:07.647266 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:07.647284 (+ 18us) server_negotiation.cc:299] Negotiation successful
0619 02:40:07.647324 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":52,"threads_started":1}
I20260619 02:40:07.647501 13517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:07.643771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.67:45923 (local address 127.0.0.1:51814)
0619 02:40:07.644039 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:07.644047 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:07.644082 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:07.644479 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:07.644479 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:07.644481 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:40:07.644512 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:07.644513 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:07.645336 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:07.645337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:07.646248 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:07.646251 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:07.647027 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:07.647028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:07.647097 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:07.647103 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 02:40:07.647134 (+ 31us) client_negotiation.cc:240] Negotiation successful
0619 02:40:07.647176 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":75,"threads_started":1}
I20260619 02:40:07.647697 13518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:07.643898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56284 (local address 127.5.241.66:36493)
0619 02:40:07.644051 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:07.644054 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:07.644062 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:40:07.644090 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:07.644091 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:07.644102 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:07.644185 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:40:07.644408 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:07.645045 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:07.646472 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:07.646930 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:07.647024 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:07.647038 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:07.647584 (+ 546us) server_negotiation.cc:299] Negotiation successful
0619 02:40:07.647625 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":70,"threads_started":1}
I20260619 02:40:07.647761 13354 tablet_service.cc:1467] Processing CreateTablet for tablet 5bf2541864a44f49bf35859df8ca3fc6 (DEFAULT_TABLE table=test-table [id=98c4d48ffb064d588cf6936a63d8c6af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:40:07.647912 13354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5bf2541864a44f49bf35859df8ca3fc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:07.647955 13516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:07.643761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:44389 (local address 127.0.0.1:33974)
0619 02:40:07.643895 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:07.643905 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:07.643962 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:07.644280 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:07.644281 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:07.644283 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:40:07.644324 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:07.644326 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:07.645256 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:07.645256 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:07.646371 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:07.646373 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:07.647115 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:07.647116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:07.647172 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:07.647175 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:40:07.647224 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 02:40:07.647259 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":4,"thread_start_us":85,"threads_started":1}
I20260619 02:40:07.647987 13218 tablet_service.cc:1467] Processing CreateTablet for tablet 5bf2541864a44f49bf35859df8ca3fc6 (DEFAULT_TABLE table=test-table [id=98c4d48ffb064d588cf6936a63d8c6af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:40:07.648089 13218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5bf2541864a44f49bf35859df8ca3fc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:07.648180 13081 tablet_service.cc:1467] Processing CreateTablet for tablet 5bf2541864a44f49bf35859df8ca3fc6 (DEFAULT_TABLE table=test-table [id=98c4d48ffb064d588cf6936a63d8c6af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:40:07.648307 13081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5bf2541864a44f49bf35859df8ca3fc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:07.649565 13523 tablet_bootstrap.cc:492] T 5bf2541864a44f49bf35859df8ca3fc6 P f277ace414904d888dfcd10ba52bc2c5: Bootstrap starting.
I20260619 02:40:07.649951 13523 tablet_bootstrap.cc:654] T 5bf2541864a44f49bf35859df8ca3fc6 P f277ace414904d888dfcd10ba52bc2c5: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:07.650205 13521 tablet_bootstrap.cc:492] T 5bf2541864a44f49bf35859df8ca3fc6 P 39f073a2df8741218340f6e1a84deb6a: Bootstrap starting.
I20260619 02:40:07.650543 13521 tablet_bootstrap.cc:654] T 5bf2541864a44f49bf35859df8ca3fc6 P 39f073a2df8741218340f6e1a84deb6a: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:07.650710 13523 tablet_bootstrap.cc:492] T 5bf2541864a44f49bf35859df8ca3fc6 P f277ace414904d888dfcd10ba52bc2c5: No bootstrap required, opened a new log
I20260619 02:40:07.650835 13523 ts_tablet_manager.cc:1397] T 5bf2541864a44f49bf35859df8ca3fc6 P f277ace414904d888dfcd10ba52bc2c5: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:40:07.650908 13522 tablet_bootstrap.cc:492] T 5bf2541864a44f49bf35859df8ca3fc6 P e90938d5560646d08f1a8872b88e0c40: Bootstrap starting.
I20260619 02:40:07.650982 13523 raft_consensus.cc:348] T 5bf2541864a44f49bf35859df8ca3fc6 P f277ace414904d888dfcd10ba52bc2c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f277ace414904d888dfcd10ba52bc2c5" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36493 } } peers { permanent_uuid: "e90938d5560646d08f1a8872b88e0c40" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 44389 } } peers { permanent_uuid: "39f073a2df8741218340f6e1a84deb6a" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 45923 } }
I20260619 02:40:07.651078 13523 raft_consensus.cc:374] T 5bf2541864a44f49bf35859df8ca3fc6 P f277ace414904d888dfcd10ba52bc2c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:07.651113 13523 raft_consensus.cc:729] T 5bf2541864a44f49bf35859df8ca3fc6 P f277ace414904d888dfcd10ba52bc2c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f277ace414904d888dfcd10ba52bc2c5, State: Initialized, Role: FOLLOWER
I20260619 02:40:07.651175 13523 consensus_queue.cc:260] T 5bf2541864a44f49bf35859df8ca3fc6 P f277ace414904d888dfcd10ba52bc2c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f277ace414904d888dfcd10ba52bc2c5" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36493 } } peers { permanent_uuid: "e90938d5560646d08f1a8872b88e0c40" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 44389 } } peers { permanent_uuid: "39f073a2df8741218340f6e1a84deb6a" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 45923 } }
I20260619 02:40:07.651244 13521 tablet_bootstrap.cc:492] T 5bf2541864a44f49bf35859df8ca3fc6 P 39f073a2df8741218340f6e1a84deb6a: No bootstrap required, opened a new log
I20260619 02:40:07.651257 13522 tablet_bootstrap.cc:654] T 5bf2541864a44f49bf35859df8ca3fc6 P e90938d5560646d08f1a8872b88e0c40: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:07.651305 13521 ts_tablet_manager.cc:1397] T 5bf2541864a44f49bf35859df8ca3fc6 P 39f073a2df8741218340f6e1a84deb6a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 02:40:07.651368 13523 ts_tablet_manager.cc:1428] T 5bf2541864a44f49bf35859df8ca3fc6 P f277ace414904d888dfcd10ba52bc2c5: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 02:40:07.651422 13521 raft_consensus.cc:348] T 5bf2541864a44f49bf35859df8ca3fc6 P 39f073a2df8741218340f6e1a84deb6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f277ace414904d888dfcd10ba52bc2c5" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36493 } } peers { permanent_uuid: "e90938d5560646d08f1a8872b88e0c40" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 44389 } } peers { permanent_uuid: "39f073a2df8741218340f6e1a84deb6a" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 45923 } }
I20260619 02:40:07.651504 13521 raft_consensus.cc:374] T 5bf2541864a44f49bf35859df8ca3fc6 P 39f073a2df8741218340f6e1a84deb6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:07.651530 13521 raft_consensus.cc:729] T 5bf2541864a44f49bf35859df8ca3fc6 P 39f073a2df8741218340f6e1a84deb6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39f073a2df8741218340f6e1a84deb6a, State: Initialized, Role: FOLLOWER
I20260619 02:40:07.651582 13521 consensus_queue.cc:260] T 5bf2541864a44f49bf35859df8ca3fc6 P 39f073a2df8741218340f6e1a84deb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f277ace414904d888dfcd10ba52bc2c5" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36493 } } peers { permanent_uuid: "e90938d5560646d08f1a8872b88e0c40" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 44389 } } peers { permanent_uuid: "39f073a2df8741218340f6e1a84deb6a" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 45923 } }
I20260619 02:40:07.651762 13521 ts_tablet_manager.cc:1428] T 5bf2541864a44f49bf35859df8ca3fc6 P 39f073a2df8741218340f6e1a84deb6a: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260619 02:40:07.651868 13502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:07.651760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45231 (local address 127.5.241.66:40871)
0619 02:40:07.651807 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:07.651838 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260619 02:40:07.652213 13501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:07.652106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45231 (local address 127.5.241.67:48643)
0619 02:40:07.652154 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:07.652180 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260619 02:40:07.652451 13522 tablet_bootstrap.cc:492] T 5bf2541864a44f49bf35859df8ca3fc6 P e90938d5560646d08f1a8872b88e0c40: No bootstrap required, opened a new log
I20260619 02:40:07.652508 13522 ts_tablet_manager.cc:1397] T 5bf2541864a44f49bf35859df8ca3fc6 P e90938d5560646d08f1a8872b88e0c40: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:40:07.652634 13522 raft_consensus.cc:348] T 5bf2541864a44f49bf35859df8ca3fc6 P e90938d5560646d08f1a8872b88e0c40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f277ace414904d888dfcd10ba52bc2c5" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36493 } } peers { permanent_uuid: "e90938d5560646d08f1a8872b88e0c40" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 44389 } } peers { permanent_uuid: "39f073a2df8741218340f6e1a84deb6a" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 45923 } }
I20260619 02:40:07.652715 13522 raft_consensus.cc:374] T 5bf2541864a44f49bf35859df8ca3fc6 P e90938d5560646d08f1a8872b88e0c40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:07.652730 13522 raft_consensus.cc:729] T 5bf2541864a44f49bf35859df8ca3fc6 P e90938d5560646d08f1a8872b88e0c40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90938d5560646d08f1a8872b88e0c40, State: Initialized, Role: FOLLOWER
I20260619 02:40:07.652750 13502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:07.652498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45231 (local address 127.5.241.66:59029)
0619 02:40:07.652689 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:07.652718 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177}
I20260619 02:40:07.652814 13522 consensus_queue.cc:260] T 5bf2541864a44f49bf35859df8ca3fc6 P e90938d5560646d08f1a8872b88e0c40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f277ace414904d888dfcd10ba52bc2c5" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36493 } } peers { permanent_uuid: "e90938d5560646d08f1a8872b88e0c40" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 44389 } } peers { permanent_uuid: "39f073a2df8741218340f6e1a84deb6a" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 45923 } }
I20260619 02:40:07.652971 13522 ts_tablet_manager.cc:1428] T 5bf2541864a44f49bf35859df8ca3fc6 P e90938d5560646d08f1a8872b88e0c40: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 02:40:07.653151 13420 heartbeater.cc:503] Master 127.5.241.124:36925 was elected leader, sending a full tablet report...
I20260619 02:40:07.653378 13503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:07.653227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45231 (local address 127.5.241.65:49559)
0619 02:40:07.653321 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:07.653349 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260619 02:40:07.653503 13501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:07.653392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45231 (local address 127.5.241.67:47639)
0619 02:40:07.653445 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:07.653473 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 02:40:07.653793 13503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:07.653706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45231 (local address 127.5.241.65:59227)
0619 02:40:07.653743 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:07.653765 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260619 02:40:07.654065 13147 heartbeater.cc:503] Master 127.5.241.124:36925 was elected leader, sending a full tablet report...
I20260619 02:40:07.689165 13528 raft_consensus.cc:482] T 5bf2541864a44f49bf35859df8ca3fc6 P 39f073a2df8741218340f6e1a84deb6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:40:07.689263 13528 raft_consensus.cc:504] T 5bf2541864a44f49bf35859df8ca3fc6 P 39f073a2df8741218340f6e1a84deb6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f277ace414904d888dfcd10ba52bc2c5" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36493 } } peers { permanent_uuid: "e90938d5560646d08f1a8872b88e0c40" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 44389 } } peers { permanent_uuid: "39f073a2df8741218340f6e1a84deb6a" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 45923 } }
I20260619 02:40:07.689404 13528 leader_election.cc:288] T 5bf2541864a44f49bf35859df8ca3fc6 P 39f073a2df8741218340f6e1a84deb6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f277ace414904d888dfcd10ba52bc2c5 (127.5.241.66:36493), e90938d5560646d08f1a8872b88e0c40 (127.5.241.65:44389)
I20260619 02:40:07.689626 13238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bf2541864a44f49bf35859df8ca3fc6" candidate_uuid: "39f073a2df8741218340f6e1a84deb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f277ace414904d888dfcd10ba52bc2c5" is_pre_election: true
I20260619 02:40:07.689720 13238 raft_consensus.cc:2454] T 5bf2541864a44f49bf35859df8ca3fc6 P f277ace414904d888dfcd10ba52bc2c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39f073a2df8741218340f6e1a84deb6a in term 0.
I20260619 02:40:07.689679 13101 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bf2541864a44f49bf35859df8ca3fc6" candidate_uuid: "39f073a2df8741218340f6e1a84deb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90938d5560646d08f1a8872b88e0c40" is_pre_election: true
I20260619 02:40:07.689792 13101 raft_consensus.cc:2454] T 5bf2541864a44f49bf35859df8ca3fc6 P e90938d5560646d08f1a8872b88e0c40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39f073a2df8741218340f6e1a84deb6a in term 0.
I20260619 02:40:07.689893 13310 leader_election.cc:302] T 5bf2541864a44f49bf35859df8ca3fc6 P 39f073a2df8741218340f6e1a84deb6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39f073a2df8741218340f6e1a84deb6a, f277ace414904d888dfcd10ba52bc2c5; no voters:
I20260619 02:40:07.689981 13528 raft_consensus.cc:2781] T 5bf2541864a44f49bf35859df8ca3fc6 P 39f073a2df8741218340f6e1a84deb6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:40:07.690011 13528 raft_consensus.cc:482] T 5bf2541864a44f49bf35859df8ca3fc6 P 39f073a2df8741218340f6e1a84deb6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:40:07.690033 13528 raft_consensus.cc:3037] T 5bf2541864a44f49bf35859df8ca3fc6 P 39f073a2df8741218340f6e1a84deb6a [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:07.691107 13528 raft_consensus.cc:504] T 5bf2541864a44f49bf35859df8ca3fc6 P 39f073a2df8741218340f6e1a84deb6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f277ace414904d888dfcd10ba52bc2c5" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36493 } } peers { permanent_uuid: "e90938d5560646d08f1a8872b88e0c40" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 44389 } } peers { permanent_uuid: "39f073a2df8741218340f6e1a84deb6a" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 45923 } }
I20260619 02:40:07.691255 13528 leader_election.cc:288] T 5bf2541864a44f49bf35859df8ca3fc6 P 39f073a2df8741218340f6e1a84deb6a [CANDIDATE]: Term 1 election: Requested vote from peers f277ace414904d888dfcd10ba52bc2c5 (127.5.241.66:36493), e90938d5560646d08f1a8872b88e0c40 (127.5.241.65:44389)
I20260619 02:40:07.691414 13101 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bf2541864a44f49bf35859df8ca3fc6" candidate_uuid: "39f073a2df8741218340f6e1a84deb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90938d5560646d08f1a8872b88e0c40"
I20260619 02:40:07.691511 13101 raft_consensus.cc:3037] T 5bf2541864a44f49bf35859df8ca3fc6 P e90938d5560646d08f1a8872b88e0c40 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:07.691444 13238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bf2541864a44f49bf35859df8ca3fc6" candidate_uuid: "39f073a2df8741218340f6e1a84deb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f277ace414904d888dfcd10ba52bc2c5"
I20260619 02:40:07.691563 13238 raft_consensus.cc:3037] T 5bf2541864a44f49bf35859df8ca3fc6 P f277ace414904d888dfcd10ba52bc2c5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:07.692198 13238 raft_consensus.cc:2454] T 5bf2541864a44f49bf35859df8ca3fc6 P f277ace414904d888dfcd10ba52bc2c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39f073a2df8741218340f6e1a84deb6a in term 1.
I20260619 02:40:07.692281 13101 raft_consensus.cc:2454] T 5bf2541864a44f49bf35859df8ca3fc6 P e90938d5560646d08f1a8872b88e0c40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39f073a2df8741218340f6e1a84deb6a in term 1.
I20260619 02:40:07.692377 13310 leader_election.cc:302] T 5bf2541864a44f49bf35859df8ca3fc6 P 39f073a2df8741218340f6e1a84deb6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39f073a2df8741218340f6e1a84deb6a, f277ace414904d888dfcd10ba52bc2c5; no voters:
I20260619 02:40:07.692505 13528 raft_consensus.cc:2781] T 5bf2541864a44f49bf35859df8ca3fc6 P 39f073a2df8741218340f6e1a84deb6a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:40:07.692684 13528 raft_consensus.cc:686] T 5bf2541864a44f49bf35859df8ca3fc6 P 39f073a2df8741218340f6e1a84deb6a [term 1 LEADER]: Becoming Leader. State: Replica: 39f073a2df8741218340f6e1a84deb6a, State: Running, Role: LEADER
I20260619 02:40:07.692734 13528 consensus_queue.cc:237] T 5bf2541864a44f49bf35859df8ca3fc6 P 39f073a2df8741218340f6e1a84deb6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f277ace414904d888dfcd10ba52bc2c5" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36493 } } peers { permanent_uuid: "e90938d5560646d08f1a8872b88e0c40" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 44389 } } peers { permanent_uuid: "39f073a2df8741218340f6e1a84deb6a" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 45923 } }
I20260619 02:40:07.693150 13501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:07.693002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45231 (local address 127.5.241.67:59591)
0619 02:40:07.693065 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:07.693111 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260619 02:40:07.693431 12966 catalog_manager.cc:5495] T 5bf2541864a44f49bf35859df8ca3fc6 P 39f073a2df8741218340f6e1a84deb6a reported cstate change: term changed from 0 to 1, leader changed from <none> to 39f073a2df8741218340f6e1a84deb6a (127.5.241.67). New cstate: current_term: 1 leader_uuid: "39f073a2df8741218340f6e1a84deb6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f277ace414904d888dfcd10ba52bc2c5" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e90938d5560646d08f1a8872b88e0c40" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 44389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39f073a2df8741218340f6e1a84deb6a" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 45923 } health_report { overall_health: HEALTHY } } }
I20260619 02:40:07.706713 13520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:07.704481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51816 (local address 127.5.241.67:45923)
0619 02:40:07.704564 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:07.704565 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:07.704573 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:40:07.704595 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:07.704595 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:07.704603 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:07.704653 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:40:07.704821 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:07.705290 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:07.705993 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:07.706187 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:07.706240 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:07.706446 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:40:07.706471 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:40:07.706509 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:07.706575 (+ 66us) server_negotiation.cc:299] Negotiation successful
0619 02:40:07.706593 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 02:40:07.707729 13238 raft_consensus.cc:1261] T 5bf2541864a44f49bf35859df8ca3fc6 P f277ace414904d888dfcd10ba52bc2c5 [term 1 FOLLOWER]: Refusing update from remote peer 39f073a2df8741218340f6e1a84deb6a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:40:07.707717 13101 raft_consensus.cc:1261] T 5bf2541864a44f49bf35859df8ca3fc6 P e90938d5560646d08f1a8872b88e0c40 [term 1 FOLLOWER]: Refusing update from remote peer 39f073a2df8741218340f6e1a84deb6a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:40:07.708015 13528 consensus_queue.cc:1035] T 5bf2541864a44f49bf35859df8ca3fc6 P 39f073a2df8741218340f6e1a84deb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f277ace414904d888dfcd10ba52bc2c5" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:40:07.708169 13530 consensus_queue.cc:1035] T 5bf2541864a44f49bf35859df8ca3fc6 P 39f073a2df8741218340f6e1a84deb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e90938d5560646d08f1a8872b88e0c40" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 44389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:40:07.708736 13284 heartbeater.cc:503] Master 127.5.241.124:36925 was elected leader, sending a full tablet report...
I20260619 02:40:07.709278 13503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:07.709165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45231 (local address 127.5.241.65:38851)
0619 02:40:07.709207 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:07.709239 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260619 02:40:07.709311 13502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:07.709155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45231 (local address 127.5.241.66:32783)
0619 02:40:07.709247 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:07.709279 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260619 02:40:07.709877 13534 mvcc.cc:205] Tried to move back new op lower bound from 7298403564369027072 to 7298403564310130688. Current Snapshot: MvccSnapshot[applied={T|T < 7298403564369027072}]
I20260619 02:40:07.710189 13501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:07.710092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45231 (local address 127.5.241.67:60553)
0619 02:40:07.710134 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:07.710161 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260619 02:40:07.710616 13502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:07.710455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45231 (local address 127.5.241.66:56351)
0619 02:40:07.710534 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:07.710588 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260619 02:40:07.711719 13501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:07.711538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45231 (local address 127.5.241.67:55241)
0619 02:40:07.711656 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:07.711696 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260619 02:40:07.712057 13503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:07.711917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45231 (local address 127.5.241.65:44675)
0619 02:40:07.711986 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:07.712021 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260619 02:40:07.712198 12966 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60420:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:40:07.712275 12966 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60420:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:40:07.713613 12966 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1a6c09a7da984a169cf84a7187990d2b: Sending DeleteTablet for 3 replicas of tablet 5bf2541864a44f49bf35859df8ca3fc6
I20260619 02:40:07.713894 13218 tablet_service.cc:1514] Processing DeleteTablet for tablet 5bf2541864a44f49bf35859df8ca3fc6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:40:07 UTC) from {username='oryx', principal='oryx/127.5.241.124@KRBTEST.COM'} at 127.0.0.1:56284
I20260619 02:40:07.713902 13354 tablet_service.cc:1514] Processing DeleteTablet for tablet 5bf2541864a44f49bf35859df8ca3fc6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:40:07 UTC) from {username='oryx', principal='oryx/127.5.241.124@KRBTEST.COM'} at 127.0.0.1:51814
I20260619 02:40:07.714010 13081 tablet_service.cc:1514] Processing DeleteTablet for tablet 5bf2541864a44f49bf35859df8ca3fc6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:40:07 UTC) from {username='oryx', principal='oryx/127.5.241.124@KRBTEST.COM'} at 127.0.0.1:33974
I20260619 02:40:07.714112 13543 tablet_replica.cc:330] T 5bf2541864a44f49bf35859df8ca3fc6 P 39f073a2df8741218340f6e1a84deb6a: stopping tablet replica
I20260619 02:40:07.714177 13543 raft_consensus.cc:2229] T 5bf2541864a44f49bf35859df8ca3fc6 P 39f073a2df8741218340f6e1a84deb6a [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:40:07.714203 13545 tablet_replica.cc:330] T 5bf2541864a44f49bf35859df8ca3fc6 P e90938d5560646d08f1a8872b88e0c40: stopping tablet replica
I20260619 02:40:07.714238 13543 raft_consensus.cc:2258] T 5bf2541864a44f49bf35859df8ca3fc6 P 39f073a2df8741218340f6e1a84deb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:40:07.714298 13545 raft_consensus.cc:2229] T 5bf2541864a44f49bf35859df8ca3fc6 P e90938d5560646d08f1a8872b88e0c40 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:40:07.714359 13545 raft_consensus.cc:2258] T 5bf2541864a44f49bf35859df8ca3fc6 P e90938d5560646d08f1a8872b88e0c40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:40:07.714437 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 12929
I20260619 02:40:07.715226 13544 tablet_replica.cc:330] T 5bf2541864a44f49bf35859df8ca3fc6 P f277ace414904d888dfcd10ba52bc2c5: stopping tablet replica
I20260619 02:40:07.715288 13544 raft_consensus.cc:2229] T 5bf2541864a44f49bf35859df8ca3fc6 P f277ace414904d888dfcd10ba52bc2c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:40:07.715337 13544 raft_consensus.cc:2258] T 5bf2541864a44f49bf35859df8ca3fc6 P f277ace414904d888dfcd10ba52bc2c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:40:07.715417 13543 ts_tablet_manager.cc:1905] T 5bf2541864a44f49bf35859df8ca3fc6 P 39f073a2df8741218340f6e1a84deb6a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:40:07.715598 13544 ts_tablet_manager.cc:1905] T 5bf2541864a44f49bf35859df8ca3fc6 P f277ace414904d888dfcd10ba52bc2c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:40:07.717093 13544 ts_tablet_manager.cc:1918] T 5bf2541864a44f49bf35859df8ca3fc6 P f277ace414904d888dfcd10ba52bc2c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:40:07.717146 13544 log.cc:1196] T 5bf2541864a44f49bf35859df8ca3fc6 P f277ace414904d888dfcd10ba52bc2c5: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-1/wal/wals/5bf2541864a44f49bf35859df8ca3fc6
I20260619 02:40:07.717341 13544 ts_tablet_manager.cc:1939] T 5bf2541864a44f49bf35859df8ca3fc6 P f277ace414904d888dfcd10ba52bc2c5: Deleting consensus metadata
I20260619 02:40:07.717573 13543 ts_tablet_manager.cc:1918] T 5bf2541864a44f49bf35859df8ca3fc6 P 39f073a2df8741218340f6e1a84deb6a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:40:07.717639 13543 log.cc:1196] T 5bf2541864a44f49bf35859df8ca3fc6 P 39f073a2df8741218340f6e1a84deb6a: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836772852275-6085-0/minicluster-data/ts-2/wal/wals/5bf2541864a44f49bf35859df8ca3fc6
I20260619 02:40:07.717684 12955 catalog_manager.cc:4841] TS f277ace414904d888dfcd10ba52bc2c5 (127.5.241.66:36493): tablet 5bf2541864a44f49bf35859df8ca3fc6 (table test-table [id=98c4d48ffb064d588cf6936a63d8c6af]) successfully deleted
I20260619 02:40:07.717895 13543 ts_tablet_manager.cc:1939] T 5bf2541864a44f49bf35859df8ca3fc6 P 39f073a2df8741218340f6e1a84deb6a: Deleting consensus metadata
I20260619 02:40:07.718366 12954 catalog_manager.cc:4841] TS 39f073a2df8741218340f6e1a84deb6a (127.5.241.67:45923): tablet 5bf2541864a44f49bf35859df8ca3fc6 (table test-table [id=98c4d48ffb064d588cf6936a63d8c6af]) successfully deleted
I20260619 02:40:07.721637 13516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:07.721454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:44389 (local address 127.0.0.1:33978)
0619 02:40:07.721530 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:07.721595 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.65:44389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260619 02:40:07.721654 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 13155
W20260619 02:40:07.722285 12955 catalog_manager.cc:4584] TS e90938d5560646d08f1a8872b88e0c40 (127.5.241.65:44389): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5bf2541864a44f49bf35859df8ca3fc6: Network error: Client connection negotiation failed: client connection to 127.5.241.65:44389: connect: Connection refused (error 111)
I20260619 02:40:07.728499 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 13291
I20260619 02:40:07.734323 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 12649
I20260619 02:40:07.739450 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 12836
2026-06-19T02:40:07Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6416 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:40:07 dist-test-slave-zpbg krb5kdc[13548](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:40:07 dist-test-slave-zpbg krb5kdc[13548](info): set up 2 sockets
Jun 19 02:40:07 dist-test-slave-zpbg krb5kdc[13548](info): commencing operation
krb5kdc: starting...
W20260619 02:40:08.783329 6085 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Jun 19 02:40:08 dist-test-slave-zpbg krb5kdc[13548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836808, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:40:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:40:08Z Disabled control of system clock
WARNING: no policy specified for oryx/127.5.241.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.241.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.241.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.126.keytab.
Entry for principal oryx/127.5.241.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.126.keytab.
I20260619 02:40:08.902292 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:36701
--webserver_interface=127.5.241.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.126.keytab
--principal=oryx/127.5.241.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:32771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:36701
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:40:08.982133 13561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:08.982378 13561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:08.982417 13561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:08.983628 13561 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:40:08.983693 13561 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 02:40:08.983718 13561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:08.983750 13561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:08.983781 13561 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:40:08.983810 13561 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:40:08.985309 13561 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:32771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:36701
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.126.keytab
--principal=oryx/127.5.241.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:36701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.241.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:40:08.985587 13561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:08.985809 13561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:40:08.990865 13561 server_base.cc:1034] running on GCE node
W20260619 02:40:08.990783 13567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:08.990806 13569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:08.990819 13566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:08.991320 13561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:08.991626 13561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:08.992759 13561 hybrid_clock.cc:648] HybridClock initialized: now 1781836808992751 us; error 32 us; skew 500 ppm
Jun 19 02:40:08 dist-test-slave-zpbg krb5kdc[13548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836808, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.241.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:08.995285 13561 init.cc:377] Logged in from keytab as oryx/127.5.241.126@KRBTEST.COM (short username oryx)
I20260619 02:40:08.995700 13561 webserver.cc:460] Webserver started at http://127.5.241.126:34723/ using document root <none> and password file <none>
I20260619 02:40:08.995915 13561 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:08.995961 13561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:08.996094 13561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:40:08.997148 13561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/master-0/data/instance:
uuid: "293499ad3ae14eb6a69f86ebd6da77d9"
format_stamp: "Formatted at 2026-06-19 02:40:08 on dist-test-slave-zpbg"
server_key: "a47453d71f776a66697ebe8899670cc5"
server_key_iv: "8857469ee222ffeb34c6e99c8535eae6"
server_key_version: "encryptionkey@0"
I20260619 02:40:08.997512 13561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/master-0/wal/instance:
uuid: "293499ad3ae14eb6a69f86ebd6da77d9"
format_stamp: "Formatted at 2026-06-19 02:40:08 on dist-test-slave-zpbg"
server_key: "a47453d71f776a66697ebe8899670cc5"
server_key_iv: "8857469ee222ffeb34c6e99c8535eae6"
server_key_version: "encryptionkey@0"
I20260619 02:40:08.998899 13561 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:40:08.999723 13576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:08.999943 13561 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:40:09.000016 13561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "293499ad3ae14eb6a69f86ebd6da77d9"
format_stamp: "Formatted at 2026-06-19 02:40:08 on dist-test-slave-zpbg"
server_key: "a47453d71f776a66697ebe8899670cc5"
server_key_iv: "8857469ee222ffeb34c6e99c8535eae6"
server_key_version: "encryptionkey@0"
I20260619 02:40:09.000108 13561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:09.013597 13561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:09.015982 13561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:09.016126 13561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:09.020892 13561 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.126:36701
I20260619 02:40:09.020973 13638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.126:36701 every 8 connection(s)
I20260619 02:40:09.021317 13561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:40:09.022037 13639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:09.024327 13639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 293499ad3ae14eb6a69f86ebd6da77d9: Bootstrap starting.
I20260619 02:40:09.024963 13639 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 293499ad3ae14eb6a69f86ebd6da77d9: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:09.025274 13639 log.cc:824] T 00000000000000000000000000000000 P 293499ad3ae14eb6a69f86ebd6da77d9: Log is configured to *not* fsync() on all Append() calls
I20260619 02:40:09.025995 13639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 293499ad3ae14eb6a69f86ebd6da77d9: No bootstrap required, opened a new log
I20260619 02:40:09.026751 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 13561
I20260619 02:40:09.026942 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/master-0/wal/instance
I20260619 02:40:09.027148 6085 external_mini_cluster.cc:1428] Setting key 8e5e79fd355d404c435494a2b34d26ef
I20260619 02:40:09.028657 13639 raft_consensus.cc:348] T 00000000000000000000000000000000 P 293499ad3ae14eb6a69f86ebd6da77d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "293499ad3ae14eb6a69f86ebd6da77d9" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 36701 } }
I20260619 02:40:09.028824 13639 raft_consensus.cc:374] T 00000000000000000000000000000000 P 293499ad3ae14eb6a69f86ebd6da77d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:09.028860 13639 raft_consensus.cc:729] T 00000000000000000000000000000000 P 293499ad3ae14eb6a69f86ebd6da77d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 293499ad3ae14eb6a69f86ebd6da77d9, State: Initialized, Role: FOLLOWER
I20260619 02:40:09.029001 13639 consensus_queue.cc:260] T 00000000000000000000000000000000 P 293499ad3ae14eb6a69f86ebd6da77d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "293499ad3ae14eb6a69f86ebd6da77d9" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 36701 } }
I20260619 02:40:09.029090 13639 raft_consensus.cc:388] T 00000000000000000000000000000000 P 293499ad3ae14eb6a69f86ebd6da77d9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:40:09.029134 13639 raft_consensus.cc:482] T 00000000000000000000000000000000 P 293499ad3ae14eb6a69f86ebd6da77d9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:40:09.029188 13639 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 293499ad3ae14eb6a69f86ebd6da77d9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:09.029912 13639 raft_consensus.cc:504] T 00000000000000000000000000000000 P 293499ad3ae14eb6a69f86ebd6da77d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "293499ad3ae14eb6a69f86ebd6da77d9" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 36701 } }
I20260619 02:40:09.030004 13639 leader_election.cc:302] T 00000000000000000000000000000000 P 293499ad3ae14eb6a69f86ebd6da77d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 293499ad3ae14eb6a69f86ebd6da77d9; no voters:
I20260619 02:40:09.030186 13639 leader_election.cc:288] T 00000000000000000000000000000000 P 293499ad3ae14eb6a69f86ebd6da77d9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:40:09.030215 13644 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 293499ad3ae14eb6a69f86ebd6da77d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:40:09.030323 13644 raft_consensus.cc:686] T 00000000000000000000000000000000 P 293499ad3ae14eb6a69f86ebd6da77d9 [term 1 LEADER]: Becoming Leader. State: Replica: 293499ad3ae14eb6a69f86ebd6da77d9, State: Running, Role: LEADER
I20260619 02:40:09.030402 13644 consensus_queue.cc:237] T 00000000000000000000000000000000 P 293499ad3ae14eb6a69f86ebd6da77d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "293499ad3ae14eb6a69f86ebd6da77d9" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 36701 } }
I20260619 02:40:09.030452 13639 sys_catalog.cc:564] T 00000000000000000000000000000000 P 293499ad3ae14eb6a69f86ebd6da77d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:40:09.030719 13644 sys_catalog.cc:455] T 00000000000000000000000000000000 P 293499ad3ae14eb6a69f86ebd6da77d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "293499ad3ae14eb6a69f86ebd6da77d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "293499ad3ae14eb6a69f86ebd6da77d9" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 36701 } } }
I20260619 02:40:09.030821 13644 sys_catalog.cc:458] T 00000000000000000000000000000000 P 293499ad3ae14eb6a69f86ebd6da77d9 [sys.catalog]: This master's current role is: LEADER
I20260619 02:40:09.030952 13644 sys_catalog.cc:455] T 00000000000000000000000000000000 P 293499ad3ae14eb6a69f86ebd6da77d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 293499ad3ae14eb6a69f86ebd6da77d9. Latest consensus state: current_term: 1 leader_uuid: "293499ad3ae14eb6a69f86ebd6da77d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "293499ad3ae14eb6a69f86ebd6da77d9" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 36701 } } }
I20260619 02:40:09.031037 13644 sys_catalog.cc:458] T 00000000000000000000000000000000 P 293499ad3ae14eb6a69f86ebd6da77d9 [sys.catalog]: This master's current role is: LEADER
I20260619 02:40:09.031172 13648 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:40:09.031919 13648 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:40:09.033574 13648 catalog_manager.cc:1347] Generated new cluster ID: 8b16d4e2bdf54147b06996fe09494bc7
I20260619 02:40:09.033643 13648 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 02:40:09 dist-test-slave-zpbg krb5kdc[13548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836808, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM
I20260619 02:40:09.039835 13643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:09.027721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43102 (local address 127.5.241.126:36701)
0619 02:40:09.027902 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:09.027911 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:09.027946 (+ 35us) server_negotiation.cc:407] Connection header received
0619 02:40:09.028637 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:09.028644 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:09.028697 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:09.028811 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:09.029677 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.031031 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.032234 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.032802 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.032912 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:09.035750 (+ 2838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:09.035754 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:09.035754 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:09.035766 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:09.037744 (+ 1978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:09.038128 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:09.038131 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:09.038131 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:09.038687 (+ 556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:09.038914 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:09.038916 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:09.038916 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:09.039233 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:09.039398 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:09.039489 (+ 91us) server_negotiation.cc:299] Negotiation successful
0619 02:40:09.039630 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":82,"threads_started":1}
I20260619 02:40:09.050006 13648 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:40:09.050630 13648 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:40:09.054304 13648 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 293499ad3ae14eb6a69f86ebd6da77d9: Generated new TSK 0
I20260619 02:40:09.054507 13648 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.5.241.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.241.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.241.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.65.keytab.
Entry for principal oryx/127.5.241.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.65.keytab.
I20260619 02:40:09.085892 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.65:0
--local_ip_for_outbound_sockets=127.5.241.65
--webserver_interface=127.5.241.65
--webserver_port=0
--tserver_master_addrs=127.5.241.126:36701
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.65.keytab
--principal=oryx/127.5.241.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:32771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:40:09.167191 13665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:09.167395 13665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:09.167423 13665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:09.168608 13665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:09.168651 13665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:09.168673 13665 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 02:40:09.168689 13665 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.65
I20260619 02:40:09.170248 13665 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:32771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.65.keytab
--principal=oryx/127.5.241.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.241.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.241.126:36701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13665
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.65
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:40:09.170475 13665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:09.170712 13665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:09.175760 13671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:09.175818 13665 server_base.cc:1034] running on GCE node
W20260619 02:40:09.175740 13673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:09.175738 13670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:09.176134 13665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:09.176394 13665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:09.177549 13665 hybrid_clock.cc:648] HybridClock initialized: now 1781836809177524 us; error 39 us; skew 500 ppm
Jun 19 02:40:09 dist-test-slave-zpbg krb5kdc[13548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836809, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.241.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:09.180282 13665 init.cc:377] Logged in from keytab as oryx/127.5.241.65@KRBTEST.COM (short username oryx)
I20260619 02:40:09.180740 13665 webserver.cc:460] Webserver started at http://127.5.241.65:33831/ using document root <none> and password file <none>
I20260619 02:40:09.180980 13665 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:09.181051 13665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:09.181202 13665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:40:09.182255 13665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-0/data/instance:
uuid: "55daaa9e103e41febd215858550495c9"
format_stamp: "Formatted at 2026-06-19 02:40:09 on dist-test-slave-zpbg"
server_key: "4e6445c1285ffe170e37bc13699b3747"
server_key_iv: "3fa76cceaef9a5e2d84e97643e218bfc"
server_key_version: "encryptionkey@0"
I20260619 02:40:09.182617 13665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-0/wal/instance:
uuid: "55daaa9e103e41febd215858550495c9"
format_stamp: "Formatted at 2026-06-19 02:40:09 on dist-test-slave-zpbg"
server_key: "4e6445c1285ffe170e37bc13699b3747"
server_key_iv: "3fa76cceaef9a5e2d84e97643e218bfc"
server_key_version: "encryptionkey@0"
I20260619 02:40:09.183961 13665 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:40:09.184736 13680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:09.184912 13665 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:40:09.185022 13665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-0/wal
uuid: "55daaa9e103e41febd215858550495c9"
format_stamp: "Formatted at 2026-06-19 02:40:09 on dist-test-slave-zpbg"
server_key: "4e6445c1285ffe170e37bc13699b3747"
server_key_iv: "3fa76cceaef9a5e2d84e97643e218bfc"
server_key_version: "encryptionkey@0"
I20260619 02:40:09.185105 13665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:09.192727 13665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:09.194670 13665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:09.194888 13665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:09.195386 13665 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:40:09.195451 13665 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:09.195488 13665 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:40:09.195516 13665 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 02:40:09 dist-test-slave-zpbg krb5kdc[13548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836809, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.241.65@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM
I20260619 02:40:09.203104 13665 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.65:46187
I20260619 02:40:09.203241 13795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.65:46187 every 8 connection(s)
I20260619 02:40:09.203562 13665 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
I20260619 02:40:09.205504 13643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:09.195912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:39277 (local address 127.5.241.126:36701)
0619 02:40:09.195994 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:09.195997 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:09.196128 (+ 131us) server_negotiation.cc:407] Connection header received
0619 02:40:09.196624 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:09.196626 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:09.196638 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:09.196718 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:09.197548 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.198412 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.199667 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.200044 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.200139 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:09.202408 (+ 2269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:09.202409 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:09.202410 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:09.202418 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:09.204225 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:09.204682 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:09.204683 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:09.204684 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:09.204735 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:09.204978 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:09.204980 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:09.204980 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:09.205101 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:09.205163 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:09.205368 (+ 205us) server_negotiation.cc:299] Negotiation successful
0619 02:40:09.205436 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 02:40:09.205667 13689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:09.195857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:36701 (local address 127.5.241.65:39277)
0619 02:40:09.196001 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:09.196042 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:09.196530 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:09.196804 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:09.196814 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:09.197151 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:09.197383 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:09.197390 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.198530 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.198531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:09.199599 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:09.199602 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.200166 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.200167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:09.200293 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:09.200828 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:09.200837 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:09.202292 (+ 1455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:09.204293 (+ 2001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:09.204298 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:09.204299 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:09.204604 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:09.204794 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:09.204795 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:09.204795 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:09.204891 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:09.205165 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:09.205168 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:09.205275 (+ 107us) client_negotiation.cc:769] Sending connection context
0619 02:40:09.205356 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 02:40:09.205473 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":74,"threads_started":1}
I20260619 02:40:09.206173 13796 heartbeater.cc:347] Connected to a master server at 127.5.241.126:36701
I20260619 02:40:09.206276 13796 heartbeater.cc:464] Registering TS with master...
I20260619 02:40:09.206450 13796 heartbeater.cc:511] Master 127.5.241.126:36701 requested a full tablet report, sending...
I20260619 02:40:09.207010 13593 ts_manager.cc:194] Registered new tserver with Master: 55daaa9e103e41febd215858550495c9 (127.5.241.65:46187)
I20260619 02:40:09.207973 13593 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.241.65@KRBTEST.COM'} at 127.5.241.65:39277
I20260619 02:40:09.210712 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 13665
I20260619 02:40:09.210853 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-0/wal/instance
I20260619 02:40:09.211099 6085 external_mini_cluster.cc:1428] Setting key 644e6feb0275d43d241d963943b11d6d
I20260619 02:40:09.213337 13643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:09.207237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:34133 (local address 127.5.241.126:36701)
0619 02:40:09.207279 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:09.207280 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:09.207395 (+ 115us) server_negotiation.cc:407] Connection header received
0619 02:40:09.207454 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:09.207455 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:09.207463 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:09.207509 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:09.207860 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.208454 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.209281 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.209454 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.209503 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:09.210367 (+ 864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:09.210368 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:09.210369 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:09.210374 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:09.212131 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:09.212482 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:09.212483 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:09.212483 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:09.212543 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:09.212724 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:09.212725 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:09.212725 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:09.212865 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:09.212932 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:09.213116 (+ 184us) server_negotiation.cc:299] Negotiation successful
0619 02:40:09.213207 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:40:09.213362 13802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:09.207176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:36701 (local address 127.5.241.65:34133)
0619 02:40:09.207294 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:09.207306 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:09.207396 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:09.207524 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:09.207525 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:09.207716 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:09.207774 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:09.207775 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.208550 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.208551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:09.209211 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:09.209213 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.209516 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.209517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:09.209564 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:09.209947 (+ 383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:09.209951 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:09.210274 (+ 323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:09.212230 (+ 1956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:09.212231 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:09.212231 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:09.212403 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:09.212610 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:09.212610 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:09.212610 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:09.212665 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:09.212961 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:09.212962 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:09.213006 (+ 44us) client_negotiation.cc:769] Sending connection context
0619 02:40:09.213075 (+ 69us) client_negotiation.cc:240] Negotiation successful
0619 02:40:09.213218 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"mutex_wait_us":45,"thread_start_us":60,"threads_started":1}
WARNING: no policy specified for oryx/127.5.241.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.241.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.241.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.66.keytab.
Entry for principal oryx/127.5.241.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.66.keytab.
I20260619 02:40:09.237985 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.66:0
--local_ip_for_outbound_sockets=127.5.241.66
--webserver_interface=127.5.241.66
--webserver_port=0
--tserver_master_addrs=127.5.241.126:36701
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.66.keytab
--principal=oryx/127.5.241.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:32771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:40:09.319036 13805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:09.319255 13805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:09.319295 13805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:09.320468 13805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:09.320535 13805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:09.320573 13805 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 02:40:09.320609 13805 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.66
I20260619 02:40:09.322090 13805 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:32771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.66.keytab
--principal=oryx/127.5.241.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.241.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.241.126:36701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13805
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.66
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:40:09.322381 13805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:09.322641 13805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:09.327534 13811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:09.327364 13813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:09.327687 13805 server_base.cc:1034] running on GCE node
W20260619 02:40:09.327862 13810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:09.328094 13805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:09.328389 13805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:09.329531 13805 hybrid_clock.cc:648] HybridClock initialized: now 1781836809329520 us; error 29 us; skew 500 ppm
Jun 19 02:40:09 dist-test-slave-zpbg krb5kdc[13548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836809, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.241.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:09.332293 13805 init.cc:377] Logged in from keytab as oryx/127.5.241.66@KRBTEST.COM (short username oryx)
I20260619 02:40:09.332849 13805 webserver.cc:460] Webserver started at http://127.5.241.66:35151/ using document root <none> and password file <none>
I20260619 02:40:09.333112 13805 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:09.333163 13805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:09.333345 13805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:40:09.334447 13805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-1/data/instance:
uuid: "766e0955e0c54660b574b4b08afd7cff"
format_stamp: "Formatted at 2026-06-19 02:40:09 on dist-test-slave-zpbg"
server_key: "74c6b3e562a7e62f324959d8ed5e3880"
server_key_iv: "b53256ba323c9c89aacb98de080a2e97"
server_key_version: "encryptionkey@0"
I20260619 02:40:09.334890 13805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-1/wal/instance:
uuid: "766e0955e0c54660b574b4b08afd7cff"
format_stamp: "Formatted at 2026-06-19 02:40:09 on dist-test-slave-zpbg"
server_key: "74c6b3e562a7e62f324959d8ed5e3880"
server_key_iv: "b53256ba323c9c89aacb98de080a2e97"
server_key_version: "encryptionkey@0"
I20260619 02:40:09.336261 13805 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:40:09.337054 13820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:09.337255 13805 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:40:09.337363 13805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-1/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-1/wal
uuid: "766e0955e0c54660b574b4b08afd7cff"
format_stamp: "Formatted at 2026-06-19 02:40:09 on dist-test-slave-zpbg"
server_key: "74c6b3e562a7e62f324959d8ed5e3880"
server_key_iv: "b53256ba323c9c89aacb98de080a2e97"
server_key_version: "encryptionkey@0"
I20260619 02:40:09.337451 13805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:09.351223 13805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:09.353238 13805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:09.353411 13805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:09.353928 13805 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:40:09.354007 13805 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:09.354075 13805 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:40:09.354105 13805 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 02:40:09 dist-test-slave-zpbg krb5kdc[13548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836809, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.241.66@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM
I20260619 02:40:09.361403 13805 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.66:36035
I20260619 02:40:09.361720 13935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.66:36035 every 8 connection(s)
I20260619 02:40:09.361819 13805 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
I20260619 02:40:09.363099 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 13805
I20260619 02:40:09.363224 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-1/wal/instance
I20260619 02:40:09.363462 6085 external_mini_cluster.cc:1428] Setting key 5eec99cf488dcc05186373f2c77412aa
I20260619 02:40:09.364295 13643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:09.354425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:60257 (local address 127.5.241.126:36701)
0619 02:40:09.354494 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:09.354497 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:09.354764 (+ 267us) server_negotiation.cc:407] Connection header received
0619 02:40:09.355533 (+ 769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:09.355534 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:09.355546 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:09.355619 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:09.356545 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.357055 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.358059 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.358272 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.358360 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:09.361156 (+ 2796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:09.361157 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:09.361158 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:09.361166 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:09.362706 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:09.363294 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:09.363295 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:09.363296 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:09.363359 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:09.363697 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:09.363698 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:09.363699 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:09.363834 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:09.363899 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:09.364153 (+ 254us) server_negotiation.cc:299] Negotiation successful
0619 02:40:09.364232 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 02:40:09.364598 13829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:09.354414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:36701 (local address 127.5.241.66:60257)
0619 02:40:09.354608 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:09.354659 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:09.355405 (+ 746us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:09.355703 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:09.355709 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:09.356127 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:09.356431 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:09.356439 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.357156 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.357157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:09.357953 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:09.357955 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.358370 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.358371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:09.358533 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:09.359099 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:09.359114 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:09.361043 (+ 1929us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:09.362818 (+ 1775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:09.362823 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:09.362825 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:09.363154 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:09.363431 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:09.363432 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:09.363433 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:09.363611 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:09.363898 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:09.363902 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:09.364018 (+ 116us) client_negotiation.cc:769] Sending connection context
0619 02:40:09.364096 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 02:40:09.364288 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":92,"threads_started":1}
I20260619 02:40:09.365067 13936 heartbeater.cc:347] Connected to a master server at 127.5.241.126:36701
I20260619 02:40:09.365155 13936 heartbeater.cc:464] Registering TS with master...
I20260619 02:40:09.365334 13936 heartbeater.cc:511] Master 127.5.241.126:36701 requested a full tablet report, sending...
I20260619 02:40:09.365711 13592 ts_manager.cc:194] Registered new tserver with Master: 766e0955e0c54660b574b4b08afd7cff (127.5.241.66:36035)
I20260619 02:40:09.366207 13592 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.241.66@KRBTEST.COM'} at 127.5.241.66:60257
I20260619 02:40:09.372460 13943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:09.366055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:36701 (local address 127.5.241.66:37621)
0619 02:40:09.366212 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:09.366223 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:09.366286 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:09.366518 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:09.366518 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:09.366757 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:09.366842 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:09.366844 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.367491 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.367492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:09.368387 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:09.368390 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.368834 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.368836 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:09.368915 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:09.369305 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:09.369310 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:09.369647 (+ 337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:09.371440 (+ 1793us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:09.371441 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:09.371441 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:09.371620 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:09.371824 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:09.371825 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:09.371825 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:09.371873 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:09.372226 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:09.372227 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:09.372263 (+ 36us) client_negotiation.cc:769] Sending connection context
0619 02:40:09.372318 (+ 55us) client_negotiation.cc:240] Negotiation successful
0619 02:40:09.372393 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":57,"threads_started":1}
I20260619 02:40:09.372538 13643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:09.366120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:37621 (local address 127.5.241.126:36701)
0619 02:40:09.366157 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:09.366158 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:09.366300 (+ 142us) server_negotiation.cc:407] Connection header received
0619 02:40:09.366411 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:09.366412 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:09.366424 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:09.366456 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:09.366924 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.367390 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.368473 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.368751 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.368806 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:09.369733 (+ 927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:09.369735 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:09.369736 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:09.369743 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:09.371352 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:09.371690 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:09.371691 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:09.371692 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:09.371754 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:09.371937 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:09.371938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:09.371938 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:09.372156 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:09.372236 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:09.372360 (+ 124us) server_negotiation.cc:299] Negotiation successful
0619 02:40:09.372439 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
WARNING: no policy specified for oryx/127.5.241.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.241.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.241.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.67.keytab.
Entry for principal oryx/127.5.241.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.67.keytab.
I20260619 02:40:09.393493 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.67:0
--local_ip_for_outbound_sockets=127.5.241.67
--webserver_interface=127.5.241.67
--webserver_port=0
--tserver_master_addrs=127.5.241.126:36701
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.67.keytab
--principal=oryx/127.5.241.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:32771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:40:09.473415 13945 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:09.473621 13945 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:09.473645 13945 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:09.474723 13945 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:09.474756 13945 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:09.474792 13945 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 02:40:09.474809 13945 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.67
I20260619 02:40:09.476204 13945 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:32771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.67.keytab
--principal=oryx/127.5.241.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.241.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.241.126:36701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13945
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.67
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:40:09.476385 13945 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:09.476562 13945 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:40:09.481588 13945 server_base.cc:1034] running on GCE node
W20260619 02:40:09.481571 13950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:09.481583 13953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:09.481571 13951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:09.482026 13945 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:09.482267 13945 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:09.483423 13945 hybrid_clock.cc:648] HybridClock initialized: now 1781836809483406 us; error 26 us; skew 500 ppm
Jun 19 02:40:09 dist-test-slave-zpbg krb5kdc[13548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836809, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.241.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:09.486006 13945 init.cc:377] Logged in from keytab as oryx/127.5.241.67@KRBTEST.COM (short username oryx)
I20260619 02:40:09.486449 13945 webserver.cc:460] Webserver started at http://127.5.241.67:40875/ using document root <none> and password file <none>
I20260619 02:40:09.486696 13945 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:09.486765 13945 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:09.486933 13945 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:40:09.488138 13945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-2/data/instance:
uuid: "bda28733e0f34538b774a2d723927573"
format_stamp: "Formatted at 2026-06-19 02:40:09 on dist-test-slave-zpbg"
server_key: "28eb54eb8ba2c138a45283e96aa3def1"
server_key_iv: "93f6c36206e2a764226acc162a2fede9"
server_key_version: "encryptionkey@0"
I20260619 02:40:09.488487 13945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-2/wal/instance:
uuid: "bda28733e0f34538b774a2d723927573"
format_stamp: "Formatted at 2026-06-19 02:40:09 on dist-test-slave-zpbg"
server_key: "28eb54eb8ba2c138a45283e96aa3def1"
server_key_iv: "93f6c36206e2a764226acc162a2fede9"
server_key_version: "encryptionkey@0"
I20260619 02:40:09.489895 13945 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:40:09.490762 13960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:09.491052 13945 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:40:09.491158 13945 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-2/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-2/wal
uuid: "bda28733e0f34538b774a2d723927573"
format_stamp: "Formatted at 2026-06-19 02:40:09 on dist-test-slave-zpbg"
server_key: "28eb54eb8ba2c138a45283e96aa3def1"
server_key_iv: "93f6c36206e2a764226acc162a2fede9"
server_key_version: "encryptionkey@0"
I20260619 02:40:09.491243 13945 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:09.512008 13945 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:09.514016 13945 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:09.514178 13945 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:09.514815 13945 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:40:09.514868 13945 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:09.514901 13945 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:40:09.514930 13945 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 02:40:09 dist-test-slave-zpbg krb5kdc[13548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836809, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.241.67@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM
I20260619 02:40:09.522073 13945 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.67:36681
I20260619 02:40:09.522219 14075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.67:36681 every 8 connection(s)
I20260619 02:40:09.522491 13945 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
I20260619 02:40:09.524987 13643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:09.515167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:51219 (local address 127.5.241.126:36701)
0619 02:40:09.515231 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:09.515233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:09.515426 (+ 193us) server_negotiation.cc:407] Connection header received
0619 02:40:09.515988 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:09.515990 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:09.516001 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:09.516068 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:09.516693 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.517275 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.518511 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.518712 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.518807 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:09.522060 (+ 3253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:09.522062 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:09.522062 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:09.522071 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:09.523668 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:09.524101 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:09.524102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:09.524102 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:09.524204 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:09.524477 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:09.524478 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:09.524478 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:09.524599 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:09.524653 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:09.524867 (+ 214us) server_negotiation.cc:299] Negotiation successful
0619 02:40:09.524926 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 02:40:09.525199 13969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:09.515151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:36701 (local address 127.5.241.67:51219)
0619 02:40:09.515303 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:09.515342 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:09.515879 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:09.516135 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:09.516139 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:09.516399 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:09.516593 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:09.516598 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.517368 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.517369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:09.518421 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:09.518424 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.518813 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.518815 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:09.518949 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:09.519515 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:09.519527 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:09.521956 (+ 2429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:09.523769 (+ 1813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:09.523775 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:09.523777 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:09.524016 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:09.524279 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:09.524280 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:09.524280 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:09.524413 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:09.524645 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:09.524649 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:09.524758 (+ 109us) client_negotiation.cc:769] Sending connection context
0619 02:40:09.524854 (+ 96us) client_negotiation.cc:240] Negotiation successful
0619 02:40:09.524981 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":72,"threads_started":1}
I20260619 02:40:09.525732 14076 heartbeater.cc:347] Connected to a master server at 127.5.241.126:36701
I20260619 02:40:09.525830 14076 heartbeater.cc:464] Registering TS with master...
I20260619 02:40:09.526033 14076 heartbeater.cc:511] Master 127.5.241.126:36701 requested a full tablet report, sending...
I20260619 02:40:09.526455 13593 ts_manager.cc:194] Registered new tserver with Master: bda28733e0f34538b774a2d723927573 (127.5.241.67:36681)
I20260619 02:40:09.526924 13593 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.241.67@KRBTEST.COM'} at 127.5.241.67:51219
I20260619 02:40:09.528473 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 13945
I20260619 02:40:09.528600 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-2/wal/instance
I20260619 02:40:09.528810 6085 external_mini_cluster.cc:1428] Setting key 02c17ec1a188eb128e78a9c34089f4db
I20260619 02:40:09.529714 6085 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:40:09.533478 13643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:09.526927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:56255 (local address 127.5.241.126:36701)
0619 02:40:09.526986 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:09.526988 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:09.527078 (+ 90us) server_negotiation.cc:407] Connection header received
0619 02:40:09.527162 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:09.527163 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:09.527172 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:09.527231 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:09.527548 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.528066 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.529202 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.529473 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.529535 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:09.530685 (+ 1150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:09.530686 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:09.530687 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:09.530693 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:09.532215 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:09.532688 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:09.532689 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:09.532690 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:09.532759 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:09.532967 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:09.532968 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:09.532969 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:09.533081 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:09.533159 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:09.533312 (+ 153us) server_negotiation.cc:299] Negotiation successful
0619 02:40:09.533388 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 02:40:09.533504 14082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:09.526865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:36701 (local address 127.5.241.67:56255)
0619 02:40:09.527018 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:09.527029 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:09.527093 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:09.527221 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:09.527221 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:09.527420 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:09.527472 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:09.527473 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.528182 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.528183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:09.529108 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:09.529110 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.529543 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.529544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:09.529614 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:09.530163 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:09.530168 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:09.530589 (+ 421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:09.532305 (+ 1716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:09.532306 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:09.532307 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:09.532563 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:09.532831 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:09.532832 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:09.532832 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:09.532892 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:09.533151 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:09.533152 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:09.533198 (+ 46us) client_negotiation.cc:769] Sending connection context
0619 02:40:09.533287 (+ 89us) client_negotiation.cc:240] Negotiation successful
0619 02:40:09.533389 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":102,"threads_started":1}
Jun 19 02:40:09 dist-test-slave-zpbg krb5kdc[13548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836809, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 02:40:09 dist-test-slave-zpbg krb5kdc[13548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:40:09 dist-test-slave-zpbg krb5kdc[13548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
W20260619 02:40:09.552903 13643 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:09.547501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43104 (local address 127.5.241.126:36701)
0619 02:40:09.547571 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:09.547574 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:09.547590 (+ 16us) server_negotiation.cc:407] Connection header received
0619 02:40:09.547615 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:09.547616 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:09.547625 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:09.547674 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:09.548211 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.548716 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.549469 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.549659 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.549713 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:09.552801 (+ 3088us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43104: BlockingRecv error: recv got EOF from 127.0.0.1:43104 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Jun 19 02:40:09 dist-test-slave-zpbg krb5kdc[13548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836809, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM
I20260619 02:40:09.560868 13643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:09.553649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43108 (local address 127.5.241.126:36701)
0619 02:40:09.553692 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:09.553694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:09.553739 (+ 45us) server_negotiation.cc:407] Connection header received
0619 02:40:09.553801 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:09.553801 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:09.553809 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:09.553844 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:09.554403 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.554983 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.555673 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.555873 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.555957 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:09.558219 (+ 2262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:09.558220 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:09.558221 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:09.558227 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:09.559746 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:09.560082 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:09.560083 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:09.560084 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:09.560148 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:09.560344 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:09.560345 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:09.560345 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:09.560462 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:09.560549 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:09.560647 (+ 98us) server_negotiation.cc:299] Negotiation successful
0619 02:40:09.560706 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 02:40:09.562587 13593 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43108:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:40:09.564206 13593 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:40:09.570935 14098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:09.566567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.67:36681 (local address 127.0.0.1:55686)
0619 02:40:09.566909 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:09.566956 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:09.567034 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:09.567700 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:09.567700 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:09.567702 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:40:09.567737 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:09.567739 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.568572 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.568574 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:09.569625 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:09.569627 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.570560 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.570561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:09.570650 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:09.570665 (+ 15us) client_negotiation.cc:769] Sending connection context
0619 02:40:09.570787 (+ 122us) client_negotiation.cc:240] Negotiation successful
0619 02:40:09.570846 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":43,"threads_started":1}
I20260619 02:40:09.570947 14099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:09.566681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55686 (local address 127.5.241.67:36681)
0619 02:40:09.567209 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:09.567213 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:09.567222 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:40:09.567258 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:09.567259 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:09.567273 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:09.567471 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:40:09.567807 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.568461 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.569721 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.570468 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.570548 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:09.570622 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:09.570797 (+ 175us) server_negotiation.cc:299] Negotiation successful
0619 02:40:09.570838 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":488,"thread_start_us":129,"threads_started":1}
I20260619 02:40:09.571609 14103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:09.566983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33812 (local address 127.5.241.65:46187)
0619 02:40:09.567189 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:09.567195 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:09.567206 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:40:09.567247 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:09.567248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:09.567263 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:09.567348 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:40:09.567848 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.569045 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.570365 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.571188 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.571256 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:09.571329 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:09.571476 (+ 147us) server_negotiation.cc:299] Negotiation successful
0619 02:40:09.571512 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":104,"threads_started":1}
I20260619 02:40:09.571612 14102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:09.566749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:46187 (local address 127.0.0.1:33812)
0619 02:40:09.567000 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:09.567011 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:09.567067 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:09.567385 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:09.567393 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:09.567395 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:40:09.567470 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:09.567472 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.569152 (+ 1680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.569153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:09.570264 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:09.570267 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.571280 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.571281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:09.571366 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:09.571370 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:40:09.571458 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 02:40:09.571515 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"mutex_wait_us":22,"thread_start_us":116,"threads_started":1}
I20260619 02:40:09.571733 14010 tablet_service.cc:1467] Processing CreateTablet for tablet 9e277829a83a49f8b4341dd663ed6430 (DEFAULT_TABLE table=test-table [id=eabaad16ac994fb4b0e0b5d05e6d83b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:40:09.572129 14010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e277829a83a49f8b4341dd663ed6430. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:09.572347 13730 tablet_service.cc:1467] Processing CreateTablet for tablet 9e277829a83a49f8b4341dd663ed6430 (DEFAULT_TABLE table=test-table [id=eabaad16ac994fb4b0e0b5d05e6d83b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:40:09.572535 14100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:09.566670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.66:36035 (local address 127.0.0.1:41762)
0619 02:40:09.566909 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:09.566956 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:09.567034 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:09.567567 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:09.567567 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:09.567569 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:40:09.567615 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:09.567617 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.569131 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.569133 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:09.570201 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:09.570204 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.572083 (+ 1879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.572084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:09.572153 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:09.572158 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 02:40:09.572415 (+ 257us) client_negotiation.cc:240] Negotiation successful
0619 02:40:09.572459 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":36,"thread_start_us":145,"threads_started":1}
I20260619 02:40:09.572785 13730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e277829a83a49f8b4341dd663ed6430. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:09.572923 14101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:09.566755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41762 (local address 127.5.241.66:36035)
0619 02:40:09.567195 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:09.567199 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:09.567208 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:40:09.567246 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:09.567247 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:09.567263 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:09.567347 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:40:09.567745 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.569007 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.570305 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.571508 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.571585 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:09.572704 (+ 1119us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:09.572739 (+ 35us) server_negotiation.cc:299] Negotiation successful
0619 02:40:09.572786 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":85,"threads_started":1}
I20260619 02:40:09.573536 13870 tablet_service.cc:1467] Processing CreateTablet for tablet 9e277829a83a49f8b4341dd663ed6430 (DEFAULT_TABLE table=test-table [id=eabaad16ac994fb4b0e0b5d05e6d83b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:40:09.573841 13870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e277829a83a49f8b4341dd663ed6430. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:09.575850 14104 tablet_bootstrap.cc:492] T 9e277829a83a49f8b4341dd663ed6430 P bda28733e0f34538b774a2d723927573: Bootstrap starting.
I20260619 02:40:09.576386 14105 tablet_bootstrap.cc:492] T 9e277829a83a49f8b4341dd663ed6430 P 55daaa9e103e41febd215858550495c9: Bootstrap starting.
I20260619 02:40:09.576591 14104 tablet_bootstrap.cc:654] T 9e277829a83a49f8b4341dd663ed6430 P bda28733e0f34538b774a2d723927573: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:09.576872 14104 log.cc:824] T 9e277829a83a49f8b4341dd663ed6430 P bda28733e0f34538b774a2d723927573: Log is configured to *not* fsync() on all Append() calls
I20260619 02:40:09.577049 14105 tablet_bootstrap.cc:654] T 9e277829a83a49f8b4341dd663ed6430 P 55daaa9e103e41febd215858550495c9: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:09.577353 14105 log.cc:824] T 9e277829a83a49f8b4341dd663ed6430 P 55daaa9e103e41febd215858550495c9: Log is configured to *not* fsync() on all Append() calls
I20260619 02:40:09.577626 14104 tablet_bootstrap.cc:492] T 9e277829a83a49f8b4341dd663ed6430 P bda28733e0f34538b774a2d723927573: No bootstrap required, opened a new log
I20260619 02:40:09.577698 14104 ts_tablet_manager.cc:1397] T 9e277829a83a49f8b4341dd663ed6430 P bda28733e0f34538b774a2d723927573: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:40:09.577785 14108 tablet_bootstrap.cc:492] T 9e277829a83a49f8b4341dd663ed6430 P 766e0955e0c54660b574b4b08afd7cff: Bootstrap starting.
I20260619 02:40:09.578159 14105 tablet_bootstrap.cc:492] T 9e277829a83a49f8b4341dd663ed6430 P 55daaa9e103e41febd215858550495c9: No bootstrap required, opened a new log
I20260619 02:40:09.578241 14105 ts_tablet_manager.cc:1397] T 9e277829a83a49f8b4341dd663ed6430 P 55daaa9e103e41febd215858550495c9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:40:09.578361 14108 tablet_bootstrap.cc:654] T 9e277829a83a49f8b4341dd663ed6430 P 766e0955e0c54660b574b4b08afd7cff: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:09.578722 14108 log.cc:824] T 9e277829a83a49f8b4341dd663ed6430 P 766e0955e0c54660b574b4b08afd7cff: Log is configured to *not* fsync() on all Append() calls
I20260619 02:40:09.579460 14108 tablet_bootstrap.cc:492] T 9e277829a83a49f8b4341dd663ed6430 P 766e0955e0c54660b574b4b08afd7cff: No bootstrap required, opened a new log
I20260619 02:40:09.579533 14108 ts_tablet_manager.cc:1397] T 9e277829a83a49f8b4341dd663ed6430 P 766e0955e0c54660b574b4b08afd7cff: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:40:09.580389 14104 raft_consensus.cc:348] T 9e277829a83a49f8b4341dd663ed6430 P bda28733e0f34538b774a2d723927573 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } } peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } }
I20260619 02:40:09.580565 14104 raft_consensus.cc:374] T 9e277829a83a49f8b4341dd663ed6430 P bda28733e0f34538b774a2d723927573 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:09.580602 14104 raft_consensus.cc:729] T 9e277829a83a49f8b4341dd663ed6430 P bda28733e0f34538b774a2d723927573 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bda28733e0f34538b774a2d723927573, State: Initialized, Role: FOLLOWER
I20260619 02:40:09.580744 14104 consensus_queue.cc:260] T 9e277829a83a49f8b4341dd663ed6430 P bda28733e0f34538b774a2d723927573 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } } peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } }
I20260619 02:40:09.581041 14104 ts_tablet_manager.cc:1428] T 9e277829a83a49f8b4341dd663ed6430 P bda28733e0f34538b774a2d723927573: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 02:40:09.581166 14076 heartbeater.cc:503] Master 127.5.241.126:36701 was elected leader, sending a full tablet report...
I20260619 02:40:09.581420 14108 raft_consensus.cc:348] T 9e277829a83a49f8b4341dd663ed6430 P 766e0955e0c54660b574b4b08afd7cff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } } peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } }
I20260619 02:40:09.581595 14108 raft_consensus.cc:374] T 9e277829a83a49f8b4341dd663ed6430 P 766e0955e0c54660b574b4b08afd7cff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:09.581638 14108 raft_consensus.cc:729] T 9e277829a83a49f8b4341dd663ed6430 P 766e0955e0c54660b574b4b08afd7cff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 766e0955e0c54660b574b4b08afd7cff, State: Initialized, Role: FOLLOWER
I20260619 02:40:09.581746 14108 consensus_queue.cc:260] T 9e277829a83a49f8b4341dd663ed6430 P 766e0955e0c54660b574b4b08afd7cff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } } peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } }
I20260619 02:40:09.582019 14108 ts_tablet_manager.cc:1428] T 9e277829a83a49f8b4341dd663ed6430 P 766e0955e0c54660b574b4b08afd7cff: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 02:40:09.582065 13936 heartbeater.cc:503] Master 127.5.241.126:36701 was elected leader, sending a full tablet report...
I20260619 02:40:09.583264 14105 raft_consensus.cc:348] T 9e277829a83a49f8b4341dd663ed6430 P 55daaa9e103e41febd215858550495c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } } peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } }
I20260619 02:40:09.583384 14105 raft_consensus.cc:374] T 9e277829a83a49f8b4341dd663ed6430 P 55daaa9e103e41febd215858550495c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:09.583403 14105 raft_consensus.cc:729] T 9e277829a83a49f8b4341dd663ed6430 P 55daaa9e103e41febd215858550495c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55daaa9e103e41febd215858550495c9, State: Initialized, Role: FOLLOWER
I20260619 02:40:09.583516 14105 consensus_queue.cc:260] T 9e277829a83a49f8b4341dd663ed6430 P 55daaa9e103e41febd215858550495c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } } peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } }
I20260619 02:40:09.583730 14105 ts_tablet_manager.cc:1428] T 9e277829a83a49f8b4341dd663ed6430 P 55daaa9e103e41febd215858550495c9: Time spent starting tablet: real 0.005s user 0.001s sys 0.002s
I20260619 02:40:09.583806 13796 heartbeater.cc:503] Master 127.5.241.126:36701 was elected leader, sending a full tablet report...
I20260619 02:40:09.608955 14111 raft_consensus.cc:482] T 9e277829a83a49f8b4341dd663ed6430 P 766e0955e0c54660b574b4b08afd7cff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:40:09.609079 14111 raft_consensus.cc:504] T 9e277829a83a49f8b4341dd663ed6430 P 766e0955e0c54660b574b4b08afd7cff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } } peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } }
I20260619 02:40:09.609477 14111 leader_election.cc:288] T 9e277829a83a49f8b4341dd663ed6430 P 766e0955e0c54660b574b4b08afd7cff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bda28733e0f34538b774a2d723927573 (127.5.241.67:36681), 55daaa9e103e41febd215858550495c9 (127.5.241.65:46187)
I20260619 02:40:09.612408 14099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:09.609813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:54565 (local address 127.5.241.67:36681)
0619 02:40:09.609872 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:09.609874 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:09.609887 (+ 13us) server_negotiation.cc:407] Connection header received
0619 02:40:09.609917 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:09.609918 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:09.609929 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:09.610016 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:40:09.610198 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.610693 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.611646 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.612015 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.612094 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:09.612104 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:09.612294 (+ 190us) server_negotiation.cc:299] Negotiation successful
0619 02:40:09.612330 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260619 02:40:09.612425 13937 tablet.cc:2369] T 9e277829a83a49f8b4341dd663ed6430 P 766e0955e0c54660b574b4b08afd7cff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:40:09.612548 14103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:09.609974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:34685 (local address 127.5.241.65:46187)
0619 02:40:09.610021 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:09.610023 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:09.610074 (+ 51us) server_negotiation.cc:407] Connection header received
0619 02:40:09.610146 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:09.610147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:09.610156 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:09.610236 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:40:09.610398 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.610920 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.611823 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.612188 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.612275 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:09.612282 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:09.612444 (+ 162us) server_negotiation.cc:299] Negotiation successful
0619 02:40:09.612488 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 02:40:09.612425 13829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:09.609732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.67:36681 (local address 127.5.241.66:54565)
0619 02:40:09.609792 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:09.609805 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:09.609849 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:09.610054 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:09.610056 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:09.610058 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:40:09.610123 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:09.610128 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.610801 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.610802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:09.611552 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:09.611554 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.612127 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.612128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:09.612206 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:09.612209 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:40:09.612284 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 02:40:09.612339 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260619 02:40:09.612747 14113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:09.609853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:46187 (local address 127.5.241.66:34685)
0619 02:40:09.610021 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:09.610031 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:09.610091 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:09.610240 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:09.610241 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:09.610242 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:40:09.610324 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:09.610326 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.611013 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.611014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:09.611730 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:09.611732 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.612275 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.612276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:09.612347 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:09.612351 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:40:09.612613 (+ 262us) client_negotiation.cc:240] Negotiation successful
0619 02:40:09.612663 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":109,"threads_started":1}
I20260619 02:40:09.612876 14030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e277829a83a49f8b4341dd663ed6430" candidate_uuid: "766e0955e0c54660b574b4b08afd7cff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bda28733e0f34538b774a2d723927573" is_pre_election: true
I20260619 02:40:09.613014 14030 raft_consensus.cc:2454] T 9e277829a83a49f8b4341dd663ed6430 P bda28733e0f34538b774a2d723927573 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 766e0955e0c54660b574b4b08afd7cff in term 0.
I20260619 02:40:09.613251 13823 leader_election.cc:302] T 9e277829a83a49f8b4341dd663ed6430 P 766e0955e0c54660b574b4b08afd7cff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 766e0955e0c54660b574b4b08afd7cff, bda28733e0f34538b774a2d723927573; no voters:
I20260619 02:40:09.613399 14111 raft_consensus.cc:2781] T 9e277829a83a49f8b4341dd663ed6430 P 766e0955e0c54660b574b4b08afd7cff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:40:09.613447 14111 raft_consensus.cc:482] T 9e277829a83a49f8b4341dd663ed6430 P 766e0955e0c54660b574b4b08afd7cff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:40:09.613389 13750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e277829a83a49f8b4341dd663ed6430" candidate_uuid: "766e0955e0c54660b574b4b08afd7cff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55daaa9e103e41febd215858550495c9" is_pre_election: true
I20260619 02:40:09.613472 14111 raft_consensus.cc:3037] T 9e277829a83a49f8b4341dd663ed6430 P 766e0955e0c54660b574b4b08afd7cff [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:09.613546 13750 raft_consensus.cc:2454] T 9e277829a83a49f8b4341dd663ed6430 P 55daaa9e103e41febd215858550495c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 766e0955e0c54660b574b4b08afd7cff in term 0.
I20260619 02:40:09.614432 14111 raft_consensus.cc:504] T 9e277829a83a49f8b4341dd663ed6430 P 766e0955e0c54660b574b4b08afd7cff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } } peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } }
I20260619 02:40:09.614562 14111 leader_election.cc:288] T 9e277829a83a49f8b4341dd663ed6430 P 766e0955e0c54660b574b4b08afd7cff [CANDIDATE]: Term 1 election: Requested vote from peers bda28733e0f34538b774a2d723927573 (127.5.241.67:36681), 55daaa9e103e41febd215858550495c9 (127.5.241.65:46187)
I20260619 02:40:09.614728 14030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e277829a83a49f8b4341dd663ed6430" candidate_uuid: "766e0955e0c54660b574b4b08afd7cff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bda28733e0f34538b774a2d723927573"
I20260619 02:40:09.614765 13750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e277829a83a49f8b4341dd663ed6430" candidate_uuid: "766e0955e0c54660b574b4b08afd7cff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55daaa9e103e41febd215858550495c9"
I20260619 02:40:09.614840 14030 raft_consensus.cc:3037] T 9e277829a83a49f8b4341dd663ed6430 P bda28733e0f34538b774a2d723927573 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:09.614884 13750 raft_consensus.cc:3037] T 9e277829a83a49f8b4341dd663ed6430 P 55daaa9e103e41febd215858550495c9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:09.615566 13750 raft_consensus.cc:2454] T 9e277829a83a49f8b4341dd663ed6430 P 55daaa9e103e41febd215858550495c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 766e0955e0c54660b574b4b08afd7cff in term 1.
I20260619 02:40:09.615698 14030 raft_consensus.cc:2454] T 9e277829a83a49f8b4341dd663ed6430 P bda28733e0f34538b774a2d723927573 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 766e0955e0c54660b574b4b08afd7cff in term 1.
I20260619 02:40:09.615747 13823 leader_election.cc:302] T 9e277829a83a49f8b4341dd663ed6430 P 766e0955e0c54660b574b4b08afd7cff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55daaa9e103e41febd215858550495c9, 766e0955e0c54660b574b4b08afd7cff; no voters:
I20260619 02:40:09.615820 14111 raft_consensus.cc:2781] T 9e277829a83a49f8b4341dd663ed6430 P 766e0955e0c54660b574b4b08afd7cff [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:40:09.615967 14111 raft_consensus.cc:686] T 9e277829a83a49f8b4341dd663ed6430 P 766e0955e0c54660b574b4b08afd7cff [term 1 LEADER]: Becoming Leader. State: Replica: 766e0955e0c54660b574b4b08afd7cff, State: Running, Role: LEADER
I20260619 02:40:09.616066 14111 consensus_queue.cc:237] T 9e277829a83a49f8b4341dd663ed6430 P 766e0955e0c54660b574b4b08afd7cff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } } peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } }
I20260619 02:40:09.616916 13592 catalog_manager.cc:5495] T 9e277829a83a49f8b4341dd663ed6430 P 766e0955e0c54660b574b4b08afd7cff reported cstate change: term changed from 0 to 1, leader changed from <none> to 766e0955e0c54660b574b4b08afd7cff (127.5.241.66). New cstate: current_term: 1 leader_uuid: "766e0955e0c54660b574b4b08afd7cff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } health_report { overall_health: HEALTHY } } }
I20260619 02:40:09.633929 14101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:09.631026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41776 (local address 127.5.241.66:36035)
0619 02:40:09.631094 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:09.631096 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:09.631107 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:40:09.631176 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:09.631177 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:09.631197 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:09.631293 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:40:09.631490 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.632172 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.633100 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.633305 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.633391 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:09.633598 (+ 207us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:40:09.633720 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:40:09.633799 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:09.633824 (+ 25us) server_negotiation.cc:299] Negotiation successful
0619 02:40:09.633843 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 02:40:09.636569 13750 raft_consensus.cc:1261] T 9e277829a83a49f8b4341dd663ed6430 P 55daaa9e103e41febd215858550495c9 [term 1 FOLLOWER]: Refusing update from remote peer 766e0955e0c54660b574b4b08afd7cff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:40:09.636897 14030 raft_consensus.cc:1261] T 9e277829a83a49f8b4341dd663ed6430 P bda28733e0f34538b774a2d723927573 [term 1 FOLLOWER]: Refusing update from remote peer 766e0955e0c54660b574b4b08afd7cff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:40:09.636914 14114 consensus_queue.cc:1035] T 9e277829a83a49f8b4341dd663ed6430 P 766e0955e0c54660b574b4b08afd7cff [LEADER]: Connected to new peer: Peer: permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:40:09.637204 14114 consensus_queue.cc:1035] T 9e277829a83a49f8b4341dd663ed6430 P 766e0955e0c54660b574b4b08afd7cff [LEADER]: Connected to new peer: Peer: permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:40:09.641999 13591 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43108:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:40:09.642107 13591 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43108:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:40:09.642896 13591 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 293499ad3ae14eb6a69f86ebd6da77d9: Sending DeleteTablet for 3 replicas of tablet 9e277829a83a49f8b4341dd663ed6430
I20260619 02:40:09.643265 14010 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e277829a83a49f8b4341dd663ed6430 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:40:09 UTC) from {username='oryx', principal='oryx/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:55686
I20260619 02:40:09.643287 13870 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e277829a83a49f8b4341dd663ed6430 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:40:09 UTC) from {username='oryx', principal='oryx/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:41762
I20260619 02:40:09.643460 14127 tablet_replica.cc:330] T 9e277829a83a49f8b4341dd663ed6430 P bda28733e0f34538b774a2d723927573: stopping tablet replica
I20260619 02:40:09.643411 13591 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43108:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260619 02:40:09.643507 13730 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e277829a83a49f8b4341dd663ed6430 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:40:09 UTC) from {username='oryx', principal='oryx/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:33812
I20260619 02:40:09.643543 14127 raft_consensus.cc:2229] T 9e277829a83a49f8b4341dd663ed6430 P bda28733e0f34538b774a2d723927573 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260619 02:40:09.643594 13591 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:40:09.643646 14127 raft_consensus.cc:2258] T 9e277829a83a49f8b4341dd663ed6430 P bda28733e0f34538b774a2d723927573 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:40:09.644016 14128 tablet_replica.cc:330] T 9e277829a83a49f8b4341dd663ed6430 P 766e0955e0c54660b574b4b08afd7cff: stopping tablet replica
I20260619 02:40:09.644155 14127 ts_tablet_manager.cc:1905] T 9e277829a83a49f8b4341dd663ed6430 P bda28733e0f34538b774a2d723927573: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:40:09.644171 14129 tablet_replica.cc:330] T 9e277829a83a49f8b4341dd663ed6430 P 55daaa9e103e41febd215858550495c9: stopping tablet replica
I20260619 02:40:09.644335 14129 raft_consensus.cc:2229] T 9e277829a83a49f8b4341dd663ed6430 P 55daaa9e103e41febd215858550495c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:40:09.644475 14128 raft_consensus.cc:2229] T 9e277829a83a49f8b4341dd663ed6430 P 766e0955e0c54660b574b4b08afd7cff [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:40:09.644649 14129 raft_consensus.cc:2258] T 9e277829a83a49f8b4341dd663ed6430 P 55daaa9e103e41febd215858550495c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:40:09.644815 14128 raft_consensus.cc:2258] T 9e277829a83a49f8b4341dd663ed6430 P 766e0955e0c54660b574b4b08afd7cff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:40:09.645136 13870 tablet_service.cc:1467] Processing CreateTablet for tablet 3a6423031a554d448584b4fd50f600d4 (DEFAULT_TABLE table=test-table [id=bcd59a01eb9545a682cec2447f3686c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:40:09.645136 13730 tablet_service.cc:1467] Processing CreateTablet for tablet 3a6423031a554d448584b4fd50f600d4 (DEFAULT_TABLE table=test-table [id=bcd59a01eb9545a682cec2447f3686c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:40:09.645251 13870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a6423031a554d448584b4fd50f600d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:09.645454 13730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a6423031a554d448584b4fd50f600d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:09.646239 14127 ts_tablet_manager.cc:1918] T 9e277829a83a49f8b4341dd663ed6430 P bda28733e0f34538b774a2d723927573: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:40:09.646302 14127 log.cc:1196] T 9e277829a83a49f8b4341dd663ed6430 P bda28733e0f34538b774a2d723927573: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-2/wal/wals/9e277829a83a49f8b4341dd663ed6430
I20260619 02:40:09.646555 14127 ts_tablet_manager.cc:1939] T 9e277829a83a49f8b4341dd663ed6430 P bda28733e0f34538b774a2d723927573: Deleting consensus metadata
I20260619 02:40:09.646718 14108 tablet_bootstrap.cc:492] T 3a6423031a554d448584b4fd50f600d4 P 766e0955e0c54660b574b4b08afd7cff: Bootstrap starting.
I20260619 02:40:09.646818 14010 tablet_service.cc:1467] Processing CreateTablet for tablet 3a6423031a554d448584b4fd50f600d4 (DEFAULT_TABLE table=test-table [id=bcd59a01eb9545a682cec2447f3686c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:40:09.646910 14010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a6423031a554d448584b4fd50f600d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:09.647156 14108 tablet_bootstrap.cc:654] T 3a6423031a554d448584b4fd50f600d4 P 766e0955e0c54660b574b4b08afd7cff: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:09.647549 14128 ts_tablet_manager.cc:1905] T 9e277829a83a49f8b4341dd663ed6430 P 766e0955e0c54660b574b4b08afd7cff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:40:09.647807 13579 catalog_manager.cc:4841] TS bda28733e0f34538b774a2d723927573 (127.5.241.67:36681): tablet 9e277829a83a49f8b4341dd663ed6430 (table test-table [id=eabaad16ac994fb4b0e0b5d05e6d83b0]) successfully deleted
I20260619 02:40:09.648200 14104 tablet_bootstrap.cc:492] T 3a6423031a554d448584b4fd50f600d4 P bda28733e0f34538b774a2d723927573: Bootstrap starting.
I20260619 02:40:09.648173 14105 tablet_bootstrap.cc:492] T 3a6423031a554d448584b4fd50f600d4 P 55daaa9e103e41febd215858550495c9: Bootstrap starting.
I20260619 02:40:09.648612 14104 tablet_bootstrap.cc:654] T 3a6423031a554d448584b4fd50f600d4 P bda28733e0f34538b774a2d723927573: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:09.648669 14105 tablet_bootstrap.cc:654] T 3a6423031a554d448584b4fd50f600d4 P 55daaa9e103e41febd215858550495c9: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:09.648106 14129 ts_tablet_manager.cc:1905] T 9e277829a83a49f8b4341dd663ed6430 P 55daaa9e103e41febd215858550495c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:40:09.649447 14128 ts_tablet_manager.cc:1918] T 9e277829a83a49f8b4341dd663ed6430 P 766e0955e0c54660b574b4b08afd7cff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:40:09.649542 14128 log.cc:1196] T 9e277829a83a49f8b4341dd663ed6430 P 766e0955e0c54660b574b4b08afd7cff: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-1/wal/wals/9e277829a83a49f8b4341dd663ed6430
I20260619 02:40:09.649777 14128 ts_tablet_manager.cc:1939] T 9e277829a83a49f8b4341dd663ed6430 P 766e0955e0c54660b574b4b08afd7cff: Deleting consensus metadata
I20260619 02:40:09.650215 13578 catalog_manager.cc:4841] TS 766e0955e0c54660b574b4b08afd7cff (127.5.241.66:36035): tablet 9e277829a83a49f8b4341dd663ed6430 (table test-table [id=eabaad16ac994fb4b0e0b5d05e6d83b0]) successfully deleted
I20260619 02:40:09.650377 14129 ts_tablet_manager.cc:1918] T 9e277829a83a49f8b4341dd663ed6430 P 55daaa9e103e41febd215858550495c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:40:09.650450 14129 log.cc:1196] T 9e277829a83a49f8b4341dd663ed6430 P 55daaa9e103e41febd215858550495c9: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-0/wal/wals/9e277829a83a49f8b4341dd663ed6430
I20260619 02:40:09.650686 14129 ts_tablet_manager.cc:1939] T 9e277829a83a49f8b4341dd663ed6430 P 55daaa9e103e41febd215858550495c9: Deleting consensus metadata
I20260619 02:40:09.651108 13579 catalog_manager.cc:4841] TS 55daaa9e103e41febd215858550495c9 (127.5.241.65:46187): tablet 9e277829a83a49f8b4341dd663ed6430 (table test-table [id=eabaad16ac994fb4b0e0b5d05e6d83b0]) successfully deleted
I20260619 02:40:09.651337 14108 tablet_bootstrap.cc:492] T 3a6423031a554d448584b4fd50f600d4 P 766e0955e0c54660b574b4b08afd7cff: No bootstrap required, opened a new log
I20260619 02:40:09.651401 14108 ts_tablet_manager.cc:1397] T 3a6423031a554d448584b4fd50f600d4 P 766e0955e0c54660b574b4b08afd7cff: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260619 02:40:09.651574 14108 raft_consensus.cc:348] T 3a6423031a554d448584b4fd50f600d4 P 766e0955e0c54660b574b4b08afd7cff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } } peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } }
I20260619 02:40:09.651656 14108 raft_consensus.cc:374] T 3a6423031a554d448584b4fd50f600d4 P 766e0955e0c54660b574b4b08afd7cff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:09.651687 14108 raft_consensus.cc:729] T 3a6423031a554d448584b4fd50f600d4 P 766e0955e0c54660b574b4b08afd7cff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 766e0955e0c54660b574b4b08afd7cff, State: Initialized, Role: FOLLOWER
I20260619 02:40:09.651736 14108 consensus_queue.cc:260] T 3a6423031a554d448584b4fd50f600d4 P 766e0955e0c54660b574b4b08afd7cff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } } peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } }
I20260619 02:40:09.651854 14108 ts_tablet_manager.cc:1428] T 3a6423031a554d448584b4fd50f600d4 P 766e0955e0c54660b574b4b08afd7cff: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:09.655157 14105 tablet_bootstrap.cc:492] T 3a6423031a554d448584b4fd50f600d4 P 55daaa9e103e41febd215858550495c9: No bootstrap required, opened a new log
I20260619 02:40:09.655182 14104 tablet_bootstrap.cc:492] T 3a6423031a554d448584b4fd50f600d4 P bda28733e0f34538b774a2d723927573: No bootstrap required, opened a new log
I20260619 02:40:09.655212 14105 ts_tablet_manager.cc:1397] T 3a6423031a554d448584b4fd50f600d4 P 55daaa9e103e41febd215858550495c9: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.000s
I20260619 02:40:09.655230 14104 ts_tablet_manager.cc:1397] T 3a6423031a554d448584b4fd50f600d4 P bda28733e0f34538b774a2d723927573: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260619 02:40:09.655364 14105 raft_consensus.cc:348] T 3a6423031a554d448584b4fd50f600d4 P 55daaa9e103e41febd215858550495c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } } peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } }
I20260619 02:40:09.655439 14105 raft_consensus.cc:374] T 3a6423031a554d448584b4fd50f600d4 P 55daaa9e103e41febd215858550495c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:09.655462 14105 raft_consensus.cc:729] T 3a6423031a554d448584b4fd50f600d4 P 55daaa9e103e41febd215858550495c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55daaa9e103e41febd215858550495c9, State: Initialized, Role: FOLLOWER
I20260619 02:40:09.655393 14104 raft_consensus.cc:348] T 3a6423031a554d448584b4fd50f600d4 P bda28733e0f34538b774a2d723927573 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } } peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } }
I20260619 02:40:09.655514 14104 raft_consensus.cc:374] T 3a6423031a554d448584b4fd50f600d4 P bda28733e0f34538b774a2d723927573 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:09.655503 14105 consensus_queue.cc:260] T 3a6423031a554d448584b4fd50f600d4 P 55daaa9e103e41febd215858550495c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } } peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } }
I20260619 02:40:09.655542 14104 raft_consensus.cc:729] T 3a6423031a554d448584b4fd50f600d4 P bda28733e0f34538b774a2d723927573 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bda28733e0f34538b774a2d723927573, State: Initialized, Role: FOLLOWER
I20260619 02:40:09.655580 14105 ts_tablet_manager.cc:1428] T 3a6423031a554d448584b4fd50f600d4 P 55daaa9e103e41febd215858550495c9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:09.655589 14104 consensus_queue.cc:260] T 3a6423031a554d448584b4fd50f600d4 P bda28733e0f34538b774a2d723927573 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } } peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } }
I20260619 02:40:09.655668 14104 ts_tablet_manager.cc:1428] T 3a6423031a554d448584b4fd50f600d4 P bda28733e0f34538b774a2d723927573: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
W20260619 02:40:09.704324 13797 tablet.cc:2369] T 3a6423031a554d448584b4fd50f600d4 P 55daaa9e103e41febd215858550495c9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:40:09.714732 14110 raft_consensus.cc:482] T 3a6423031a554d448584b4fd50f600d4 P bda28733e0f34538b774a2d723927573 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:40:09.714891 14110 raft_consensus.cc:504] T 3a6423031a554d448584b4fd50f600d4 P bda28733e0f34538b774a2d723927573 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } } peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } }
I20260619 02:40:09.715246 14110 leader_election.cc:288] T 3a6423031a554d448584b4fd50f600d4 P bda28733e0f34538b774a2d723927573 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 55daaa9e103e41febd215858550495c9 (127.5.241.65:46187), 766e0955e0c54660b574b4b08afd7cff (127.5.241.66:36035)
I20260619 02:40:09.718883 13969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:09.715429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:46187 (local address 127.5.241.67:51735)
0619 02:40:09.715497 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:09.715511 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:09.715559 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:09.715803 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:09.715803 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:09.715805 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:40:09.715903 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:09.715905 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.716777 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.716778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:09.717878 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:09.717881 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.718585 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.718586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:09.718672 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:09.718676 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:40:09.718730 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 02:40:09.718805 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":9}
I20260619 02:40:09.718888 14103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:09.715607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:51735 (local address 127.5.241.65:46187)
0619 02:40:09.715653 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:09.715655 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:09.715663 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:40:09.715694 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:09.715695 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:09.715705 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:09.715753 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:40:09.715970 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.716691 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.718001 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.718484 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.718577 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:09.718587 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:09.718738 (+ 151us) server_negotiation.cc:299] Negotiation successful
0619 02:40:09.718817 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 02:40:09.718940 14101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:09.715518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:47535 (local address 127.5.241.66:36035)
0619 02:40:09.715562 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:09.715564 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:09.715743 (+ 179us) server_negotiation.cc:407] Connection header received
0619 02:40:09.715778 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:09.715779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:09.715791 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:09.715858 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:40:09.716040 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.716524 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.717988 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.718558 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.718632 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:09.718641 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:09.718825 (+ 184us) server_negotiation.cc:299] Negotiation successful
0619 02:40:09.718867 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 02:40:09.719177 13750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a6423031a554d448584b4fd50f600d4" candidate_uuid: "bda28733e0f34538b774a2d723927573" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55daaa9e103e41febd215858550495c9" is_pre_election: true
I20260619 02:40:09.719275 13750 raft_consensus.cc:2454] T 3a6423031a554d448584b4fd50f600d4 P 55daaa9e103e41febd215858550495c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bda28733e0f34538b774a2d723927573 in term 0.
I20260619 02:40:09.719342 14133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:09.715429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.66:36035 (local address 127.5.241.67:47535)
0619 02:40:09.715664 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:09.715675 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:09.715730 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:09.715870 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:09.715870 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:09.715872 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:40:09.715943 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:09.715945 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.716632 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.716633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:09.717879 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:09.717881 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.718650 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.718651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:09.718715 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:09.718719 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:40:09.719180 (+ 461us) client_negotiation.cc:240] Negotiation successful
0619 02:40:09.719223 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"mutex_wait_us":119,"thread_start_us":58,"threads_started":1}
I20260619 02:40:09.719457 13963 leader_election.cc:302] T 3a6423031a554d448584b4fd50f600d4 P bda28733e0f34538b774a2d723927573 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55daaa9e103e41febd215858550495c9, bda28733e0f34538b774a2d723927573; no voters:
I20260619 02:40:09.719590 14110 raft_consensus.cc:2781] T 3a6423031a554d448584b4fd50f600d4 P bda28733e0f34538b774a2d723927573 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:40:09.719637 14110 raft_consensus.cc:482] T 3a6423031a554d448584b4fd50f600d4 P bda28733e0f34538b774a2d723927573 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:40:09.719663 14110 raft_consensus.cc:3037] T 3a6423031a554d448584b4fd50f600d4 P bda28733e0f34538b774a2d723927573 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:09.719600 13890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a6423031a554d448584b4fd50f600d4" candidate_uuid: "bda28733e0f34538b774a2d723927573" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "766e0955e0c54660b574b4b08afd7cff" is_pre_election: true
I20260619 02:40:09.719709 13890 raft_consensus.cc:2454] T 3a6423031a554d448584b4fd50f600d4 P 766e0955e0c54660b574b4b08afd7cff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bda28733e0f34538b774a2d723927573 in term 0.
I20260619 02:40:09.720444 14110 raft_consensus.cc:504] T 3a6423031a554d448584b4fd50f600d4 P bda28733e0f34538b774a2d723927573 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } } peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } }
I20260619 02:40:09.720599 14110 leader_election.cc:288] T 3a6423031a554d448584b4fd50f600d4 P bda28733e0f34538b774a2d723927573 [CANDIDATE]: Term 1 election: Requested vote from peers 55daaa9e103e41febd215858550495c9 (127.5.241.65:46187), 766e0955e0c54660b574b4b08afd7cff (127.5.241.66:36035)
I20260619 02:40:09.720726 13750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a6423031a554d448584b4fd50f600d4" candidate_uuid: "bda28733e0f34538b774a2d723927573" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55daaa9e103e41febd215858550495c9"
I20260619 02:40:09.720795 13750 raft_consensus.cc:3037] T 3a6423031a554d448584b4fd50f600d4 P 55daaa9e103e41febd215858550495c9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:09.720819 13890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a6423031a554d448584b4fd50f600d4" candidate_uuid: "bda28733e0f34538b774a2d723927573" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "766e0955e0c54660b574b4b08afd7cff"
I20260619 02:40:09.720898 13890 raft_consensus.cc:3037] T 3a6423031a554d448584b4fd50f600d4 P 766e0955e0c54660b574b4b08afd7cff [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:09.721415 13750 raft_consensus.cc:2454] T 3a6423031a554d448584b4fd50f600d4 P 55daaa9e103e41febd215858550495c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bda28733e0f34538b774a2d723927573 in term 1.
I20260619 02:40:09.721568 13963 leader_election.cc:302] T 3a6423031a554d448584b4fd50f600d4 P bda28733e0f34538b774a2d723927573 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55daaa9e103e41febd215858550495c9, bda28733e0f34538b774a2d723927573; no voters:
I20260619 02:40:09.721659 14110 raft_consensus.cc:2781] T 3a6423031a554d448584b4fd50f600d4 P bda28733e0f34538b774a2d723927573 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:40:09.721758 13890 raft_consensus.cc:2454] T 3a6423031a554d448584b4fd50f600d4 P 766e0955e0c54660b574b4b08afd7cff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bda28733e0f34538b774a2d723927573 in term 1.
I20260619 02:40:09.721810 14110 raft_consensus.cc:686] T 3a6423031a554d448584b4fd50f600d4 P bda28733e0f34538b774a2d723927573 [term 1 LEADER]: Becoming Leader. State: Replica: bda28733e0f34538b774a2d723927573, State: Running, Role: LEADER
I20260619 02:40:09.721875 14110 consensus_queue.cc:237] T 3a6423031a554d448584b4fd50f600d4 P bda28733e0f34538b774a2d723927573 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } } peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } }
I20260619 02:40:09.722576 13591 catalog_manager.cc:5495] T 3a6423031a554d448584b4fd50f600d4 P bda28733e0f34538b774a2d723927573 reported cstate change: term changed from 0 to 1, leader changed from <none> to bda28733e0f34538b774a2d723927573 (127.5.241.67). New cstate: current_term: 1 leader_uuid: "bda28733e0f34538b774a2d723927573" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } health_report { overall_health: HEALTHY } } }
Jun 19 02:40:09 dist-test-slave-zpbg krb5kdc[13548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836808, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.241.126@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM
I20260619 02:40:09.770934 13643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:09.763565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43116 (local address 127.5.241.126:36701)
0619 02:40:09.763671 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:09.763673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:09.763776 (+ 103us) server_negotiation.cc:407] Connection header received
0619 02:40:09.763871 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:09.763872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:09.763883 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:09.763939 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:09.764437 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.764937 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.765730 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.765911 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.765983 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:09.768172 (+ 2189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:09.768173 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:09.768174 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:09.768180 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:09.769533 (+ 1353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:09.769928 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:09.769929 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:09.769929 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:09.769981 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:09.770243 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:09.770244 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:09.770244 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:09.770359 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:09.770454 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:09.770688 (+ 234us) server_negotiation.cc:299] Negotiation successful
0619 02:40:09.770864 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":74}
I20260619 02:40:09.770933 14141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:09.763480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:36701 (local address 127.0.0.1:43116)
0619 02:40:09.763693 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:09.763711 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:09.763797 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:09.763948 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:09.763949 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:09.764280 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:09.764326 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:09.764328 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.765016 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.765018 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:09.765638 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:09.765640 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:09.765980 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:09.765981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:09.766031 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:09.766564 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:09.766569 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:09.768074 (+ 1505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:09.769619 (+ 1545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:09.769626 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:09.769628 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:09.769858 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:09.770073 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:09.770074 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:09.770074 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:09.770153 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:09.770476 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:09.770483 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:09.770544 (+ 61us) client_negotiation.cc:769] Sending connection context
0619 02:40:09.770630 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 02:40:09.770744 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":112,"threads_started":1}
W20260619 02:40:09.773052 14077 tablet.cc:2369] T 3a6423031a554d448584b4fd50f600d4 P bda28733e0f34538b774a2d723927573: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:40:09.773155 13591 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:43116:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 02:40:09.773360 13591 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:40:09.774843 13730 tablet_service.cc:1467] Processing CreateTablet for tablet 339c19c479ec4555b771caf79d151521 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d644acbfe3a6492686a2f7ad17466fa2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 02:40:09.774866 13870 tablet_service.cc:1467] Processing CreateTablet for tablet 339c19c479ec4555b771caf79d151521 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d644acbfe3a6492686a2f7ad17466fa2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 02:40:09.774884 14010 tablet_service.cc:1467] Processing CreateTablet for tablet 339c19c479ec4555b771caf79d151521 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d644acbfe3a6492686a2f7ad17466fa2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 02:40:09.775002 13870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 339c19c479ec4555b771caf79d151521. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:09.775009 13730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 339c19c479ec4555b771caf79d151521. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:09.775048 14010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 339c19c479ec4555b771caf79d151521. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:09.776312 14104 tablet_bootstrap.cc:492] T 339c19c479ec4555b771caf79d151521 P bda28733e0f34538b774a2d723927573: Bootstrap starting.
I20260619 02:40:09.776423 14108 tablet_bootstrap.cc:492] T 339c19c479ec4555b771caf79d151521 P 766e0955e0c54660b574b4b08afd7cff: Bootstrap starting.
I20260619 02:40:09.776707 14104 tablet_bootstrap.cc:654] T 339c19c479ec4555b771caf79d151521 P bda28733e0f34538b774a2d723927573: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:09.776733 14108 tablet_bootstrap.cc:654] T 339c19c479ec4555b771caf79d151521 P 766e0955e0c54660b574b4b08afd7cff: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:09.776981 14105 tablet_bootstrap.cc:492] T 339c19c479ec4555b771caf79d151521 P 55daaa9e103e41febd215858550495c9: Bootstrap starting.
I20260619 02:40:09.777235 14108 tablet_bootstrap.cc:492] T 339c19c479ec4555b771caf79d151521 P 766e0955e0c54660b574b4b08afd7cff: No bootstrap required, opened a new log
I20260619 02:40:09.777302 14108 ts_tablet_manager.cc:1397] T 339c19c479ec4555b771caf79d151521 P 766e0955e0c54660b574b4b08afd7cff: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:40:09.777346 14105 tablet_bootstrap.cc:654] T 339c19c479ec4555b771caf79d151521 P 55daaa9e103e41febd215858550495c9: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:09.777417 14108 raft_consensus.cc:348] T 339c19c479ec4555b771caf79d151521 P 766e0955e0c54660b574b4b08afd7cff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } } peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } }
I20260619 02:40:09.777506 14108 raft_consensus.cc:374] T 339c19c479ec4555b771caf79d151521 P 766e0955e0c54660b574b4b08afd7cff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:09.777530 14108 raft_consensus.cc:729] T 339c19c479ec4555b771caf79d151521 P 766e0955e0c54660b574b4b08afd7cff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 766e0955e0c54660b574b4b08afd7cff, State: Initialized, Role: FOLLOWER
I20260619 02:40:09.777587 14108 consensus_queue.cc:260] T 339c19c479ec4555b771caf79d151521 P 766e0955e0c54660b574b4b08afd7cff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } } peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } }
I20260619 02:40:09.777691 14108 ts_tablet_manager.cc:1428] T 339c19c479ec4555b771caf79d151521 P 766e0955e0c54660b574b4b08afd7cff: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:09.777705 14114 tablet_replica.cc:439] T 339c19c479ec4555b771caf79d151521 P 766e0955e0c54660b574b4b08afd7cff: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } } peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } } }
I20260619 02:40:09.777778 14114 tablet_replica.cc:442] T 339c19c479ec4555b771caf79d151521 P 766e0955e0c54660b574b4b08afd7cff: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:40:09.778188 14104 tablet_bootstrap.cc:492] T 339c19c479ec4555b771caf79d151521 P bda28733e0f34538b774a2d723927573: No bootstrap required, opened a new log
I20260619 02:40:09.778242 14104 ts_tablet_manager.cc:1397] T 339c19c479ec4555b771caf79d151521 P bda28733e0f34538b774a2d723927573: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:40:09.778277 14105 tablet_bootstrap.cc:492] T 339c19c479ec4555b771caf79d151521 P 55daaa9e103e41febd215858550495c9: No bootstrap required, opened a new log
I20260619 02:40:09.778319 14105 ts_tablet_manager.cc:1397] T 339c19c479ec4555b771caf79d151521 P 55daaa9e103e41febd215858550495c9: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260619 02:40:09.778394 14104 raft_consensus.cc:348] T 339c19c479ec4555b771caf79d151521 P bda28733e0f34538b774a2d723927573 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } } peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } }
I20260619 02:40:09.778467 14104 raft_consensus.cc:374] T 339c19c479ec4555b771caf79d151521 P bda28733e0f34538b774a2d723927573 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:09.778486 14104 raft_consensus.cc:729] T 339c19c479ec4555b771caf79d151521 P bda28733e0f34538b774a2d723927573 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bda28733e0f34538b774a2d723927573, State: Initialized, Role: FOLLOWER
I20260619 02:40:09.778453 14105 raft_consensus.cc:348] T 339c19c479ec4555b771caf79d151521 P 55daaa9e103e41febd215858550495c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } } peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } }
I20260619 02:40:09.778549 14105 raft_consensus.cc:374] T 339c19c479ec4555b771caf79d151521 P 55daaa9e103e41febd215858550495c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:09.778580 14105 raft_consensus.cc:729] T 339c19c479ec4555b771caf79d151521 P 55daaa9e103e41febd215858550495c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55daaa9e103e41febd215858550495c9, State: Initialized, Role: FOLLOWER
I20260619 02:40:09.778565 14104 consensus_queue.cc:260] T 339c19c479ec4555b771caf79d151521 P bda28733e0f34538b774a2d723927573 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } } peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } }
I20260619 02:40:09.778653 14104 ts_tablet_manager.cc:1428] T 339c19c479ec4555b771caf79d151521 P bda28733e0f34538b774a2d723927573: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:09.778635 14105 consensus_queue.cc:260] T 339c19c479ec4555b771caf79d151521 P 55daaa9e103e41febd215858550495c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } } peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } }
I20260619 02:40:09.778729 14105 ts_tablet_manager.cc:1428] T 339c19c479ec4555b771caf79d151521 P 55daaa9e103e41febd215858550495c9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:09.778699 14110 tablet_replica.cc:439] T 339c19c479ec4555b771caf79d151521 P bda28733e0f34538b774a2d723927573: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } } peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } } }
I20260619 02:40:09.778820 14110 tablet_replica.cc:442] T 339c19c479ec4555b771caf79d151521 P bda28733e0f34538b774a2d723927573: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:40:09.778846 14112 tablet_replica.cc:439] T 339c19c479ec4555b771caf79d151521 P 55daaa9e103e41febd215858550495c9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } } peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } } }
I20260619 02:40:09.778913 14112 tablet_replica.cc:442] T 339c19c479ec4555b771caf79d151521 P 55daaa9e103e41febd215858550495c9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:40:10.114874 14110 consensus_queue.cc:1035] T 3a6423031a554d448584b4fd50f600d4 P bda28733e0f34538b774a2d723927573 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:40:10.116981 14146 consensus_queue.cc:1035] T 3a6423031a554d448584b4fd50f600d4 P bda28733e0f34538b774a2d723927573 [LEADER]: Connected to new peer: Peer: permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:40:10.130201 14114 raft_consensus.cc:482] T 339c19c479ec4555b771caf79d151521 P 766e0955e0c54660b574b4b08afd7cff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:40:10.130299 14114 raft_consensus.cc:504] T 339c19c479ec4555b771caf79d151521 P 766e0955e0c54660b574b4b08afd7cff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } } peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } }
I20260619 02:40:10.130455 14114 leader_election.cc:288] T 339c19c479ec4555b771caf79d151521 P 766e0955e0c54660b574b4b08afd7cff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 55daaa9e103e41febd215858550495c9 (127.5.241.65:46187), bda28733e0f34538b774a2d723927573 (127.5.241.67:36681)
I20260619 02:40:10.130649 13750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "339c19c479ec4555b771caf79d151521" candidate_uuid: "766e0955e0c54660b574b4b08afd7cff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55daaa9e103e41febd215858550495c9" is_pre_election: true
I20260619 02:40:10.130668 14030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "339c19c479ec4555b771caf79d151521" candidate_uuid: "766e0955e0c54660b574b4b08afd7cff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bda28733e0f34538b774a2d723927573" is_pre_election: true
I20260619 02:40:10.130755 13750 raft_consensus.cc:2454] T 339c19c479ec4555b771caf79d151521 P 55daaa9e103e41febd215858550495c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 766e0955e0c54660b574b4b08afd7cff in term 0.
I20260619 02:40:10.130817 14030 raft_consensus.cc:2454] T 339c19c479ec4555b771caf79d151521 P bda28733e0f34538b774a2d723927573 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 766e0955e0c54660b574b4b08afd7cff in term 0.
I20260619 02:40:10.130977 13823 leader_election.cc:302] T 339c19c479ec4555b771caf79d151521 P 766e0955e0c54660b574b4b08afd7cff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 766e0955e0c54660b574b4b08afd7cff, bda28733e0f34538b774a2d723927573; no voters:
I20260619 02:40:10.131099 14114 raft_consensus.cc:2781] T 339c19c479ec4555b771caf79d151521 P 766e0955e0c54660b574b4b08afd7cff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:40:10.131165 14114 raft_consensus.cc:482] T 339c19c479ec4555b771caf79d151521 P 766e0955e0c54660b574b4b08afd7cff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:40:10.131197 14114 raft_consensus.cc:3037] T 339c19c479ec4555b771caf79d151521 P 766e0955e0c54660b574b4b08afd7cff [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:10.131947 14114 raft_consensus.cc:504] T 339c19c479ec4555b771caf79d151521 P 766e0955e0c54660b574b4b08afd7cff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } } peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } }
I20260619 02:40:10.132083 14114 leader_election.cc:288] T 339c19c479ec4555b771caf79d151521 P 766e0955e0c54660b574b4b08afd7cff [CANDIDATE]: Term 1 election: Requested vote from peers 55daaa9e103e41febd215858550495c9 (127.5.241.65:46187), bda28733e0f34538b774a2d723927573 (127.5.241.67:36681)
I20260619 02:40:10.132304 14030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "339c19c479ec4555b771caf79d151521" candidate_uuid: "766e0955e0c54660b574b4b08afd7cff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bda28733e0f34538b774a2d723927573"
I20260619 02:40:10.132385 14030 raft_consensus.cc:3037] T 339c19c479ec4555b771caf79d151521 P bda28733e0f34538b774a2d723927573 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:10.133036 14030 raft_consensus.cc:2454] T 339c19c479ec4555b771caf79d151521 P bda28733e0f34538b774a2d723927573 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 766e0955e0c54660b574b4b08afd7cff in term 1.
I20260619 02:40:10.133147 13750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "339c19c479ec4555b771caf79d151521" candidate_uuid: "766e0955e0c54660b574b4b08afd7cff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55daaa9e103e41febd215858550495c9"
I20260619 02:40:10.133255 13750 raft_consensus.cc:3037] T 339c19c479ec4555b771caf79d151521 P 55daaa9e103e41febd215858550495c9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:10.133235 13823 leader_election.cc:302] T 339c19c479ec4555b771caf79d151521 P 766e0955e0c54660b574b4b08afd7cff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 766e0955e0c54660b574b4b08afd7cff, bda28733e0f34538b774a2d723927573; no voters:
I20260619 02:40:10.133347 14114 raft_consensus.cc:2781] T 339c19c479ec4555b771caf79d151521 P 766e0955e0c54660b574b4b08afd7cff [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:40:10.133402 14114 raft_consensus.cc:686] T 339c19c479ec4555b771caf79d151521 P 766e0955e0c54660b574b4b08afd7cff [term 1 LEADER]: Becoming Leader. State: Replica: 766e0955e0c54660b574b4b08afd7cff, State: Running, Role: LEADER
I20260619 02:40:10.133453 14114 consensus_queue.cc:237] T 339c19c479ec4555b771caf79d151521 P 766e0955e0c54660b574b4b08afd7cff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } } peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } }
I20260619 02:40:10.133704 14122 tablet_replica.cc:439] T 339c19c479ec4555b771caf79d151521 P 766e0955e0c54660b574b4b08afd7cff: TxnStatusTablet state changed. Reason: New leader 766e0955e0c54660b574b4b08afd7cff. Latest consensus state: current_term: 1 leader_uuid: "766e0955e0c54660b574b4b08afd7cff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } } peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } } }
I20260619 02:40:10.133783 14122 tablet_replica.cc:442] T 339c19c479ec4555b771caf79d151521 P 766e0955e0c54660b574b4b08afd7cff: This TxnStatusTablet replica's current role is: LEADER
I20260619 02:40:10.133921 14149 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 02:40:10.134037 14149 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 02:40:10.134057 13750 raft_consensus.cc:2454] T 339c19c479ec4555b771caf79d151521 P 55daaa9e103e41febd215858550495c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 766e0955e0c54660b574b4b08afd7cff in term 1.
I20260619 02:40:10.134011 13592 catalog_manager.cc:5495] T 339c19c479ec4555b771caf79d151521 P 766e0955e0c54660b574b4b08afd7cff reported cstate change: term changed from 0 to 1, leader changed from <none> to 766e0955e0c54660b574b4b08afd7cff (127.5.241.66). New cstate: current_term: 1 leader_uuid: "766e0955e0c54660b574b4b08afd7cff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } health_report { overall_health: UNKNOWN } } }
I20260619 02:40:10.220795 14101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:10.218255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41792 (local address 127.5.241.66:36035)
0619 02:40:10.218306 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:10.218308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:10.218318 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:40:10.218373 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:10.218374 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:10.218385 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:10.218439 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:40:10.218574 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:10.219191 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:10.220037 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:10.220251 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:10.220337 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:10.220522 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:40:10.220561 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:40:10.220627 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:10.220694 (+ 67us) server_negotiation.cc:299] Negotiation successful
0619 02:40:10.220715 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 02:40:10.220813 14141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:10.218172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.66:36035 (local address 127.0.0.1:41792)
0619 02:40:10.218243 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:10.218256 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:10.218306 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:10.218453 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:10.218453 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:10.218454 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:40:10.218505 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:10.218508 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:10.219263 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:10.219264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:10.219939 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:10.219943 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:10.220340 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:10.220341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:10.220407 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:10.220421 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:40:10.220628 (+ 207us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:40:10.220636 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 02:40:10.220689 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 02:40:10.220737 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 02:40:10.221745 14030 raft_consensus.cc:1261] T 339c19c479ec4555b771caf79d151521 P bda28733e0f34538b774a2d723927573 [term 1 FOLLOWER]: Refusing update from remote peer 766e0955e0c54660b574b4b08afd7cff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:40:10.221796 13750 raft_consensus.cc:1261] T 339c19c479ec4555b771caf79d151521 P 55daaa9e103e41febd215858550495c9 [term 1 FOLLOWER]: Refusing update from remote peer 766e0955e0c54660b574b4b08afd7cff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:40:10.222000 14114 consensus_queue.cc:1035] T 339c19c479ec4555b771caf79d151521 P 766e0955e0c54660b574b4b08afd7cff [LEADER]: Connected to new peer: Peer: permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:40:10.222138 14122 consensus_queue.cc:1035] T 339c19c479ec4555b771caf79d151521 P 766e0955e0c54660b574b4b08afd7cff [LEADER]: Connected to new peer: Peer: permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:40:10.222654 14112 tablet_replica.cc:439] T 339c19c479ec4555b771caf79d151521 P 55daaa9e103e41febd215858550495c9: TxnStatusTablet state changed. Reason: New leader 766e0955e0c54660b574b4b08afd7cff. Latest consensus state: current_term: 1 leader_uuid: "766e0955e0c54660b574b4b08afd7cff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } } peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } } }
I20260619 02:40:10.222680 14146 tablet_replica.cc:439] T 339c19c479ec4555b771caf79d151521 P bda28733e0f34538b774a2d723927573: TxnStatusTablet state changed. Reason: New leader 766e0955e0c54660b574b4b08afd7cff. Latest consensus state: current_term: 1 leader_uuid: "766e0955e0c54660b574b4b08afd7cff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } } peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } } }
I20260619 02:40:10.222752 14112 tablet_replica.cc:442] T 339c19c479ec4555b771caf79d151521 P 55daaa9e103e41febd215858550495c9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:40:10.222790 14146 tablet_replica.cc:442] T 339c19c479ec4555b771caf79d151521 P bda28733e0f34538b774a2d723927573: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:40:10.223644 14114 tablet_replica.cc:439] T 339c19c479ec4555b771caf79d151521 P 766e0955e0c54660b574b4b08afd7cff: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "766e0955e0c54660b574b4b08afd7cff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } } peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } } }
I20260619 02:40:10.223730 14114 tablet_replica.cc:442] T 339c19c479ec4555b771caf79d151521 P 766e0955e0c54660b574b4b08afd7cff: This TxnStatusTablet replica's current role is: LEADER
I20260619 02:40:10.223786 14114 tablet_replica.cc:439] T 339c19c479ec4555b771caf79d151521 P 766e0955e0c54660b574b4b08afd7cff: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "766e0955e0c54660b574b4b08afd7cff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } } peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } } }
I20260619 02:40:10.223821 14114 tablet_replica.cc:442] T 339c19c479ec4555b771caf79d151521 P 766e0955e0c54660b574b4b08afd7cff: This TxnStatusTablet replica's current role is: LEADER
I20260619 02:40:10.223956 14119 mvcc.cc:205] Tried to move back new op lower bound from 7298403574666514432 to 7298403574306844672. Current Snapshot: MvccSnapshot[applied={T|T < 7298403574666514432}]
I20260619 02:40:10.224066 14146 tablet_replica.cc:439] T 339c19c479ec4555b771caf79d151521 P bda28733e0f34538b774a2d723927573: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "766e0955e0c54660b574b4b08afd7cff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } } peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } } }
I20260619 02:40:10.224135 14146 tablet_replica.cc:442] T 339c19c479ec4555b771caf79d151521 P bda28733e0f34538b774a2d723927573: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:40:10.224114 14117 mvcc.cc:205] Tried to move back new op lower bound from 7298403574666514432 to 7298403574306844672. Current Snapshot: MvccSnapshot[applied={T|T < 7298403574666514432 or (T in {7298403574666514432})}]
I20260619 02:40:10.224203 14118 mvcc.cc:205] Tried to move back new op lower bound from 7298403574666514432 to 7298403574306844672. Current Snapshot: MvccSnapshot[applied={T|T < 7298403574666514432}]
I20260619 02:40:10.224361 14112 tablet_replica.cc:439] T 339c19c479ec4555b771caf79d151521 P 55daaa9e103e41febd215858550495c9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "766e0955e0c54660b574b4b08afd7cff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } } peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } } }
I20260619 02:40:10.224429 14112 tablet_replica.cc:442] T 339c19c479ec4555b771caf79d151521 P 55daaa9e103e41febd215858550495c9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:40:10.228724 14160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:10.225928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55694 (local address 127.5.241.67:36681)
0619 02:40:10.226034 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:10.226036 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:10.226045 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:40:10.226121 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:10.226122 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:10.226141 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:10.226236 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:40:10.226343 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:10.226890 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:10.227887 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:10.228111 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:10.228198 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:10.228371 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:40:10.228483 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:40:10.228562 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:10.228613 (+ 51us) server_negotiation.cc:299] Negotiation successful
0619 02:40:10.228637 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":63,"threads_started":1}
I20260619 02:40:10.233711 14101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:10.231317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:39229 (local address 127.5.241.66:36035)
0619 02:40:10.231357 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:10.231359 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:10.231514 (+ 155us) server_negotiation.cc:407] Connection header received
0619 02:40:10.231591 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:10.231592 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:10.231604 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:10.231661 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:40:10.231754 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:10.232211 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:10.233004 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:10.233197 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:10.233276 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:10.233447 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:40:10.233481 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:40:10.233556 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:10.233619 (+ 63us) server_negotiation.cc:299] Negotiation successful
0619 02:40:10.233639 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 02:40:10.233728 14163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:10.231249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.66:36035 (local address 127.5.241.67:39229)
0619 02:40:10.231459 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:10.231470 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:10.231529 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:10.231663 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:10.231664 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:10.231665 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:40:10.231708 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:10.231709 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:10.232326 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:10.232327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:10.232908 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:10.232909 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:10.233285 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:10.233286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:10.233339 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:10.233359 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:40:10.233559 (+ 200us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:40:10.233561 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 02:40:10.233614 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 02:40:10.233659 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":87,"threads_started":1}
I20260619 02:40:10.245558 14160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:10.240104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:54975 (local address 127.5.241.67:36681)
0619 02:40:10.240142 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:10.240144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:10.242957 (+ 2813us) server_negotiation.cc:407] Connection header received
0619 02:40:10.243000 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:10.243001 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:10.243010 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:10.243060 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:40:10.243213 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:10.243731 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:10.244847 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:10.245026 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:10.245082 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:10.245303 (+ 221us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:40:10.245334 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:40:10.245390 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:10.245450 (+ 60us) server_negotiation.cc:299] Negotiation successful
0619 02:40:10.245503 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 02:40:10.245584 14165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:10.240067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.67:36681 (local address 127.5.241.66:54975)
0619 02:40:10.242878 (+ 2811us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:10.242891 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:10.242954 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:10.243078 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:10.243093 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:10.243095 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:40:10.243146 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:10.243148 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:10.243809 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:10.243810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:10.244763 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:10.244765 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:10.245107 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:10.245108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:10.245168 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:10.245184 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:40:10.245404 (+ 220us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:40:10.245406 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 02:40:10.245449 (+ 43us) client_negotiation.cc:240] Negotiation successful
0619 02:40:10.245496 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2771,"thread_start_us":51,"threads_started":1}
I20260619 02:40:10.255924 13592 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43108:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:40:10.256067 13592 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43108:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:40:10.256729 13592 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 293499ad3ae14eb6a69f86ebd6da77d9: Sending DeleteTablet for 3 replicas of tablet 3a6423031a554d448584b4fd50f600d4
I20260619 02:40:10.257006 13870 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a6423031a554d448584b4fd50f600d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:40:10 UTC) from {username='oryx', principal='oryx/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:41762
I20260619 02:40:10.257014 13730 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a6423031a554d448584b4fd50f600d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:40:10 UTC) from {username='oryx', principal='oryx/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:33812
I20260619 02:40:10.257180 14010 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a6423031a554d448584b4fd50f600d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:40:10 UTC) from {username='oryx', principal='oryx/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:55686
I20260619 02:40:10.257215 14168 tablet_replica.cc:330] T 3a6423031a554d448584b4fd50f600d4 P 55daaa9e103e41febd215858550495c9: stopping tablet replica
I20260619 02:40:10.257277 14168 raft_consensus.cc:2229] T 3a6423031a554d448584b4fd50f600d4 P 55daaa9e103e41febd215858550495c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:40:10.257333 14168 raft_consensus.cc:2258] T 3a6423031a554d448584b4fd50f600d4 P 55daaa9e103e41febd215858550495c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:40:10.257305 14169 tablet_replica.cc:330] T 3a6423031a554d448584b4fd50f600d4 P 766e0955e0c54660b574b4b08afd7cff: stopping tablet replica
I20260619 02:40:10.257477 14169 raft_consensus.cc:2229] T 3a6423031a554d448584b4fd50f600d4 P 766e0955e0c54660b574b4b08afd7cff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:40:10.257550 14169 raft_consensus.cc:2258] T 3a6423031a554d448584b4fd50f600d4 P 766e0955e0c54660b574b4b08afd7cff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:40:10.257601 14168 ts_tablet_manager.cc:1905] T 3a6423031a554d448584b4fd50f600d4 P 55daaa9e103e41febd215858550495c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:40:10.257840 14169 ts_tablet_manager.cc:1905] T 3a6423031a554d448584b4fd50f600d4 P 766e0955e0c54660b574b4b08afd7cff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:40:10.259588 14169 ts_tablet_manager.cc:1918] T 3a6423031a554d448584b4fd50f600d4 P 766e0955e0c54660b574b4b08afd7cff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:40:10.259585 14168 ts_tablet_manager.cc:1918] T 3a6423031a554d448584b4fd50f600d4 P 55daaa9e103e41febd215858550495c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:40:10.259654 14168 log.cc:1196] T 3a6423031a554d448584b4fd50f600d4 P 55daaa9e103e41febd215858550495c9: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-0/wal/wals/3a6423031a554d448584b4fd50f600d4
I20260619 02:40:10.259771 14169 log.cc:1196] T 3a6423031a554d448584b4fd50f600d4 P 766e0955e0c54660b574b4b08afd7cff: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-1/wal/wals/3a6423031a554d448584b4fd50f600d4
I20260619 02:40:10.259907 14168 ts_tablet_manager.cc:1939] T 3a6423031a554d448584b4fd50f600d4 P 55daaa9e103e41febd215858550495c9: Deleting consensus metadata
I20260619 02:40:10.260138 14169 ts_tablet_manager.cc:1939] T 3a6423031a554d448584b4fd50f600d4 P 766e0955e0c54660b574b4b08afd7cff: Deleting consensus metadata
I20260619 02:40:10.260694 14170 tablet_replica.cc:330] T 3a6423031a554d448584b4fd50f600d4 P bda28733e0f34538b774a2d723927573: stopping tablet replica
I20260619 02:40:10.260747 13578 catalog_manager.cc:4841] TS 766e0955e0c54660b574b4b08afd7cff (127.5.241.66:36035): tablet 3a6423031a554d448584b4fd50f600d4 (table test-table [id=bcd59a01eb9545a682cec2447f3686c1]) successfully deleted
I20260619 02:40:10.260797 14170 raft_consensus.cc:2229] T 3a6423031a554d448584b4fd50f600d4 P bda28733e0f34538b774a2d723927573 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:40:10.260952 14170 raft_consensus.cc:2258] T 3a6423031a554d448584b4fd50f600d4 P bda28733e0f34538b774a2d723927573 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:40:10.261319 13579 catalog_manager.cc:4841] TS 55daaa9e103e41febd215858550495c9 (127.5.241.65:46187): tablet 3a6423031a554d448584b4fd50f600d4 (table test-table [id=bcd59a01eb9545a682cec2447f3686c1]) successfully deleted
I20260619 02:40:10.261363 14170 ts_tablet_manager.cc:1905] T 3a6423031a554d448584b4fd50f600d4 P bda28733e0f34538b774a2d723927573: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:40:10.263091 14170 ts_tablet_manager.cc:1918] T 3a6423031a554d448584b4fd50f600d4 P bda28733e0f34538b774a2d723927573: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:40:10.263160 14170 log.cc:1196] T 3a6423031a554d448584b4fd50f600d4 P bda28733e0f34538b774a2d723927573: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-2/wal/wals/3a6423031a554d448584b4fd50f600d4
I20260619 02:40:10.263406 14170 ts_tablet_manager.cc:1939] T 3a6423031a554d448584b4fd50f600d4 P bda28733e0f34538b774a2d723927573: Deleting consensus metadata
I20260619 02:40:10.263748 13579 catalog_manager.cc:4841] TS bda28733e0f34538b774a2d723927573 (127.5.241.67:36681): tablet 3a6423031a554d448584b4fd50f600d4 (table test-table [id=bcd59a01eb9545a682cec2447f3686c1]) successfully deleted
Jun 19 02:40:10 dist-test-slave-zpbg krb5kdc[13548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836810, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 02:40:10 dist-test-slave-zpbg krb5kdc[13548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:40:10 dist-test-slave-zpbg krb5kdc[13548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
W20260619 02:40:10.285128 14178 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:10.278845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43124 (local address 127.5.241.126:36701)
0619 02:40:10.279070 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:10.279073 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:10.279107 (+ 34us) server_negotiation.cc:407] Connection header received
0619 02:40:10.279145 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:10.279146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:10.279160 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:10.279227 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:10.279829 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:10.280640 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:10.281459 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:10.281773 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:10.281856 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:10.285044 (+ 3188us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43124: BlockingRecv error: recv got EOF from 127.0.0.1:43124 (error 108)
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":71,"threads_started":1}
Jun 19 02:40:10 dist-test-slave-zpbg krb5kdc[13548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836810, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM
I20260619 02:40:10.293715 14178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:10.285957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43138 (local address 127.5.241.126:36701)
0619 02:40:10.286002 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:10.286004 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:10.286060 (+ 56us) server_negotiation.cc:407] Connection header received
0619 02:40:10.286139 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:10.286140 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:10.286151 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:10.286195 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:10.286666 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:10.287211 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:10.288080 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:10.288277 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:10.288347 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:10.290750 (+ 2403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:10.290751 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:10.290752 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:10.290759 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:10.292493 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:10.292846 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:10.292847 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:10.292847 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:10.292933 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:10.293165 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:10.293166 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:10.293167 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:10.293307 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:10.293419 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:10.293520 (+ 101us) server_negotiation.cc:299] Negotiation successful
0619 02:40:10.293606 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 02:40:10.295539 13592 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43138:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:40:10.295735 13592 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:40:10.297178 13730 tablet_service.cc:1467] Processing CreateTablet for tablet e8e6af8ae3c94e6d8723112c5b99cd48 (DEFAULT_TABLE table=test-table [id=f38324ac106e4085800ee6fec0ed9cf8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:40:10.297201 14010 tablet_service.cc:1467] Processing CreateTablet for tablet e8e6af8ae3c94e6d8723112c5b99cd48 (DEFAULT_TABLE table=test-table [id=f38324ac106e4085800ee6fec0ed9cf8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:40:10.297322 14010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8e6af8ae3c94e6d8723112c5b99cd48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:10.297322 13730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8e6af8ae3c94e6d8723112c5b99cd48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:10.298714 14184 tablet_bootstrap.cc:492] T e8e6af8ae3c94e6d8723112c5b99cd48 P bda28733e0f34538b774a2d723927573: Bootstrap starting.
I20260619 02:40:10.298763 13870 tablet_service.cc:1467] Processing CreateTablet for tablet e8e6af8ae3c94e6d8723112c5b99cd48 (DEFAULT_TABLE table=test-table [id=f38324ac106e4085800ee6fec0ed9cf8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:40:10.298928 13870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8e6af8ae3c94e6d8723112c5b99cd48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:10.299365 14184 tablet_bootstrap.cc:654] T e8e6af8ae3c94e6d8723112c5b99cd48 P bda28733e0f34538b774a2d723927573: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:10.300037 14184 tablet_bootstrap.cc:492] T e8e6af8ae3c94e6d8723112c5b99cd48 P bda28733e0f34538b774a2d723927573: No bootstrap required, opened a new log
I20260619 02:40:10.300096 14184 ts_tablet_manager.cc:1397] T e8e6af8ae3c94e6d8723112c5b99cd48 P bda28733e0f34538b774a2d723927573: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:40:10.300308 14187 tablet_bootstrap.cc:492] T e8e6af8ae3c94e6d8723112c5b99cd48 P 766e0955e0c54660b574b4b08afd7cff: Bootstrap starting.
I20260619 02:40:10.300367 14185 tablet_bootstrap.cc:492] T e8e6af8ae3c94e6d8723112c5b99cd48 P 55daaa9e103e41febd215858550495c9: Bootstrap starting.
I20260619 02:40:10.300278 14184 raft_consensus.cc:348] T e8e6af8ae3c94e6d8723112c5b99cd48 P bda28733e0f34538b774a2d723927573 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } } peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } }
I20260619 02:40:10.300521 14184 raft_consensus.cc:374] T e8e6af8ae3c94e6d8723112c5b99cd48 P bda28733e0f34538b774a2d723927573 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:10.300554 14184 raft_consensus.cc:729] T e8e6af8ae3c94e6d8723112c5b99cd48 P bda28733e0f34538b774a2d723927573 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bda28733e0f34538b774a2d723927573, State: Initialized, Role: FOLLOWER
I20260619 02:40:10.300621 14184 consensus_queue.cc:260] T e8e6af8ae3c94e6d8723112c5b99cd48 P bda28733e0f34538b774a2d723927573 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } } peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } }
I20260619 02:40:10.300814 14185 tablet_bootstrap.cc:654] T e8e6af8ae3c94e6d8723112c5b99cd48 P 55daaa9e103e41febd215858550495c9: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:10.301293 14187 tablet_bootstrap.cc:654] T e8e6af8ae3c94e6d8723112c5b99cd48 P 766e0955e0c54660b574b4b08afd7cff: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:10.301347 14184 ts_tablet_manager.cc:1428] T e8e6af8ae3c94e6d8723112c5b99cd48 P bda28733e0f34538b774a2d723927573: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:40:10.301832 14187 tablet_bootstrap.cc:492] T e8e6af8ae3c94e6d8723112c5b99cd48 P 766e0955e0c54660b574b4b08afd7cff: No bootstrap required, opened a new log
I20260619 02:40:10.301898 14187 ts_tablet_manager.cc:1397] T e8e6af8ae3c94e6d8723112c5b99cd48 P 766e0955e0c54660b574b4b08afd7cff: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:40:10.302012 14187 raft_consensus.cc:348] T e8e6af8ae3c94e6d8723112c5b99cd48 P 766e0955e0c54660b574b4b08afd7cff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } } peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } }
I20260619 02:40:10.302096 14187 raft_consensus.cc:374] T e8e6af8ae3c94e6d8723112c5b99cd48 P 766e0955e0c54660b574b4b08afd7cff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:10.302160 14187 raft_consensus.cc:729] T e8e6af8ae3c94e6d8723112c5b99cd48 P 766e0955e0c54660b574b4b08afd7cff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 766e0955e0c54660b574b4b08afd7cff, State: Initialized, Role: FOLLOWER
I20260619 02:40:10.302232 14187 consensus_queue.cc:260] T e8e6af8ae3c94e6d8723112c5b99cd48 P 766e0955e0c54660b574b4b08afd7cff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } } peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } }
I20260619 02:40:10.302353 14187 ts_tablet_manager.cc:1428] T e8e6af8ae3c94e6d8723112c5b99cd48 P 766e0955e0c54660b574b4b08afd7cff: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:10.307246 14185 tablet_bootstrap.cc:492] T e8e6af8ae3c94e6d8723112c5b99cd48 P 55daaa9e103e41febd215858550495c9: No bootstrap required, opened a new log
I20260619 02:40:10.307296 14185 ts_tablet_manager.cc:1397] T e8e6af8ae3c94e6d8723112c5b99cd48 P 55daaa9e103e41febd215858550495c9: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260619 02:40:10.307461 14185 raft_consensus.cc:348] T e8e6af8ae3c94e6d8723112c5b99cd48 P 55daaa9e103e41febd215858550495c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } } peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } }
I20260619 02:40:10.307523 14185 raft_consensus.cc:374] T e8e6af8ae3c94e6d8723112c5b99cd48 P 55daaa9e103e41febd215858550495c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:10.307542 14185 raft_consensus.cc:729] T e8e6af8ae3c94e6d8723112c5b99cd48 P 55daaa9e103e41febd215858550495c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55daaa9e103e41febd215858550495c9, State: Initialized, Role: FOLLOWER
I20260619 02:40:10.307596 14185 consensus_queue.cc:260] T e8e6af8ae3c94e6d8723112c5b99cd48 P 55daaa9e103e41febd215858550495c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } } peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } }
I20260619 02:40:10.307713 14185 ts_tablet_manager.cc:1428] T e8e6af8ae3c94e6d8723112c5b99cd48 P 55daaa9e103e41febd215858550495c9: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 02:40:10.409144 14112 raft_consensus.cc:482] T e8e6af8ae3c94e6d8723112c5b99cd48 P 55daaa9e103e41febd215858550495c9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:40:10.409250 14112 raft_consensus.cc:504] T e8e6af8ae3c94e6d8723112c5b99cd48 P 55daaa9e103e41febd215858550495c9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } } peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } }
I20260619 02:40:10.409704 14112 leader_election.cc:288] T e8e6af8ae3c94e6d8723112c5b99cd48 P 55daaa9e103e41febd215858550495c9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 766e0955e0c54660b574b4b08afd7cff (127.5.241.66:36035), bda28733e0f34538b774a2d723927573 (127.5.241.67:36681)
I20260619 02:40:10.413830 14160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:10.410085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:58321 (local address 127.5.241.67:36681)
0619 02:40:10.410130 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:10.410132 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:10.410142 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:40:10.410221 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:10.410222 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:10.410240 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:10.410311 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:40:10.410606 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:10.411378 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:10.412963 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:10.413451 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:10.413518 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:10.413526 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:10.413701 (+ 175us) server_negotiation.cc:299] Negotiation successful
0619 02:40:10.413748 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 02:40:10.413849 14190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:10.409964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.67:36681 (local address 127.5.241.65:58321)
0619 02:40:10.410088 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:10.410096 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:10.410149 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:10.410357 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:10.410358 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:10.410362 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:40:10.410491 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:10.410493 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:10.411489 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:10.411490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:10.412813 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:10.412816 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:10.413536 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:10.413538 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:10.413619 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:10.413625 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 02:40:10.413680 (+ 55us) client_negotiation.cc:240] Negotiation successful
0619 02:40:10.413734 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":13,"thread_start_us":50,"threads_started":1}
I20260619 02:40:10.413937 14101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:10.409992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:55027 (local address 127.5.241.66:36035)
0619 02:40:10.410049 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:10.410051 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:10.410150 (+ 99us) server_negotiation.cc:407] Connection header received
0619 02:40:10.410243 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:10.410244 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:10.410254 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:10.410311 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:40:10.410605 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:10.411378 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:10.413044 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:10.413563 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:10.413638 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:10.413648 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:10.413821 (+ 173us) server_negotiation.cc:299] Negotiation successful
0619 02:40:10.413867 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 02:40:10.414135 14191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:10.409910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.66:36035 (local address 127.5.241.65:55027)
0619 02:40:10.410069 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:10.410083 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:10.410169 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:10.410357 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:10.410358 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:10.410362 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:40:10.410491 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:10.410493 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:10.411845 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:10.411846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:10.412918 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:10.412920 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:10.413644 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:10.413646 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:10.413741 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:10.413746 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 02:40:10.413804 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 02:40:10.413850 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":34,"thread_start_us":100,"threads_started":1}
I20260619 02:40:10.414180 14030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8e6af8ae3c94e6d8723112c5b99cd48" candidate_uuid: "55daaa9e103e41febd215858550495c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bda28733e0f34538b774a2d723927573" is_pre_election: true
I20260619 02:40:10.414289 14030 raft_consensus.cc:2454] T e8e6af8ae3c94e6d8723112c5b99cd48 P bda28733e0f34538b774a2d723927573 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55daaa9e103e41febd215858550495c9 in term 0.
I20260619 02:40:10.414491 13683 leader_election.cc:302] T e8e6af8ae3c94e6d8723112c5b99cd48 P 55daaa9e103e41febd215858550495c9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55daaa9e103e41febd215858550495c9, bda28733e0f34538b774a2d723927573; no voters:
I20260619 02:40:10.414522 13890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8e6af8ae3c94e6d8723112c5b99cd48" candidate_uuid: "55daaa9e103e41febd215858550495c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "766e0955e0c54660b574b4b08afd7cff" is_pre_election: true
I20260619 02:40:10.414597 13890 raft_consensus.cc:2454] T e8e6af8ae3c94e6d8723112c5b99cd48 P 766e0955e0c54660b574b4b08afd7cff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55daaa9e103e41febd215858550495c9 in term 0.
I20260619 02:40:10.414655 14112 raft_consensus.cc:2781] T e8e6af8ae3c94e6d8723112c5b99cd48 P 55daaa9e103e41febd215858550495c9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:40:10.414705 14112 raft_consensus.cc:482] T e8e6af8ae3c94e6d8723112c5b99cd48 P 55daaa9e103e41febd215858550495c9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:40:10.414731 14112 raft_consensus.cc:3037] T e8e6af8ae3c94e6d8723112c5b99cd48 P 55daaa9e103e41febd215858550495c9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:10.415491 14112 raft_consensus.cc:504] T e8e6af8ae3c94e6d8723112c5b99cd48 P 55daaa9e103e41febd215858550495c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } } peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } }
I20260619 02:40:10.415642 14112 leader_election.cc:288] T e8e6af8ae3c94e6d8723112c5b99cd48 P 55daaa9e103e41febd215858550495c9 [CANDIDATE]: Term 1 election: Requested vote from peers 766e0955e0c54660b574b4b08afd7cff (127.5.241.66:36035), bda28733e0f34538b774a2d723927573 (127.5.241.67:36681)
I20260619 02:40:10.415805 13890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8e6af8ae3c94e6d8723112c5b99cd48" candidate_uuid: "55daaa9e103e41febd215858550495c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "766e0955e0c54660b574b4b08afd7cff"
I20260619 02:40:10.415812 14030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8e6af8ae3c94e6d8723112c5b99cd48" candidate_uuid: "55daaa9e103e41febd215858550495c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bda28733e0f34538b774a2d723927573"
I20260619 02:40:10.415875 14030 raft_consensus.cc:3037] T e8e6af8ae3c94e6d8723112c5b99cd48 P bda28733e0f34538b774a2d723927573 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:10.415875 13890 raft_consensus.cc:3037] T e8e6af8ae3c94e6d8723112c5b99cd48 P 766e0955e0c54660b574b4b08afd7cff [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:10.416630 13890 raft_consensus.cc:2454] T e8e6af8ae3c94e6d8723112c5b99cd48 P 766e0955e0c54660b574b4b08afd7cff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55daaa9e103e41febd215858550495c9 in term 1.
I20260619 02:40:10.416628 14030 raft_consensus.cc:2454] T e8e6af8ae3c94e6d8723112c5b99cd48 P bda28733e0f34538b774a2d723927573 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55daaa9e103e41febd215858550495c9 in term 1.
I20260619 02:40:10.416802 13683 leader_election.cc:302] T e8e6af8ae3c94e6d8723112c5b99cd48 P 55daaa9e103e41febd215858550495c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 55daaa9e103e41febd215858550495c9, 766e0955e0c54660b574b4b08afd7cff, bda28733e0f34538b774a2d723927573; no voters:
I20260619 02:40:10.416929 14112 raft_consensus.cc:2781] T e8e6af8ae3c94e6d8723112c5b99cd48 P 55daaa9e103e41febd215858550495c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:40:10.417101 14112 raft_consensus.cc:686] T e8e6af8ae3c94e6d8723112c5b99cd48 P 55daaa9e103e41febd215858550495c9 [term 1 LEADER]: Becoming Leader. State: Replica: 55daaa9e103e41febd215858550495c9, State: Running, Role: LEADER
I20260619 02:40:10.417188 14112 consensus_queue.cc:237] T e8e6af8ae3c94e6d8723112c5b99cd48 P 55daaa9e103e41febd215858550495c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } } peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } }
I20260619 02:40:10.417945 13592 catalog_manager.cc:5495] T e8e6af8ae3c94e6d8723112c5b99cd48 P 55daaa9e103e41febd215858550495c9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 55daaa9e103e41febd215858550495c9 (127.5.241.65). New cstate: current_term: 1 leader_uuid: "55daaa9e103e41febd215858550495c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } health_report { overall_health: UNKNOWN } } }
I20260619 02:40:10.447103 14195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:10.443959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33814 (local address 127.5.241.65:46187)
0619 02:40:10.444251 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:10.444253 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:10.444264 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:40:10.444320 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:10.444321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:10.444344 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:10.444425 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:40:10.444719 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:10.445235 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:10.446081 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:10.446287 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:10.446344 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:10.446569 (+ 225us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:40:10.446681 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:40:10.446749 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:10.446909 (+ 160us) server_negotiation.cc:299] Negotiation successful
0619 02:40:10.446932 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":238,"threads_started":1}
I20260619 02:40:10.448220 14030 raft_consensus.cc:1261] T e8e6af8ae3c94e6d8723112c5b99cd48 P bda28733e0f34538b774a2d723927573 [term 1 FOLLOWER]: Refusing update from remote peer 55daaa9e103e41febd215858550495c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:40:10.448235 13890 raft_consensus.cc:1261] T e8e6af8ae3c94e6d8723112c5b99cd48 P 766e0955e0c54660b574b4b08afd7cff [term 1 FOLLOWER]: Refusing update from remote peer 55daaa9e103e41febd215858550495c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:40:10.448518 14112 consensus_queue.cc:1035] T e8e6af8ae3c94e6d8723112c5b99cd48 P 55daaa9e103e41febd215858550495c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:40:10.448655 14192 consensus_queue.cc:1035] T e8e6af8ae3c94e6d8723112c5b99cd48 P 55daaa9e103e41febd215858550495c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:40:10.452251 13592 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:40:10.452339 13592 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:40:10.452926 13592 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 293499ad3ae14eb6a69f86ebd6da77d9: Sending DeleteTablet for 3 replicas of tablet e8e6af8ae3c94e6d8723112c5b99cd48
I20260619 02:40:10.453205 13730 tablet_service.cc:1514] Processing DeleteTablet for tablet e8e6af8ae3c94e6d8723112c5b99cd48 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:40:10 UTC) from {username='oryx', principal='oryx/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:33812
I20260619 02:40:10.453212 14010 tablet_service.cc:1514] Processing DeleteTablet for tablet e8e6af8ae3c94e6d8723112c5b99cd48 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:40:10 UTC) from {username='oryx', principal='oryx/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:55686
I20260619 02:40:10.453303 14168 tablet_replica.cc:330] T e8e6af8ae3c94e6d8723112c5b99cd48 P 55daaa9e103e41febd215858550495c9: stopping tablet replica
I20260619 02:40:10.453303 14170 tablet_replica.cc:330] T e8e6af8ae3c94e6d8723112c5b99cd48 P bda28733e0f34538b774a2d723927573: stopping tablet replica
I20260619 02:40:10.453353 14168 raft_consensus.cc:2229] T e8e6af8ae3c94e6d8723112c5b99cd48 P 55daaa9e103e41febd215858550495c9 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:40:10.453353 14170 raft_consensus.cc:2229] T e8e6af8ae3c94e6d8723112c5b99cd48 P bda28733e0f34538b774a2d723927573 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:40:10.453400 14170 raft_consensus.cc:2258] T e8e6af8ae3c94e6d8723112c5b99cd48 P bda28733e0f34538b774a2d723927573 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:40:10.453433 14168 raft_consensus.cc:2258] T e8e6af8ae3c94e6d8723112c5b99cd48 P 55daaa9e103e41febd215858550495c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:40:10.453749 14170 ts_tablet_manager.cc:1905] T e8e6af8ae3c94e6d8723112c5b99cd48 P bda28733e0f34538b774a2d723927573: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:40:10.453792 14168 ts_tablet_manager.cc:1905] T e8e6af8ae3c94e6d8723112c5b99cd48 P 55daaa9e103e41febd215858550495c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:40:10.455416 14170 ts_tablet_manager.cc:1918] T e8e6af8ae3c94e6d8723112c5b99cd48 P bda28733e0f34538b774a2d723927573: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:40:10.455421 13870 tablet_service.cc:1514] Processing DeleteTablet for tablet e8e6af8ae3c94e6d8723112c5b99cd48 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:40:10 UTC) from {username='oryx', principal='oryx/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:41762
I20260619 02:40:10.455478 14170 log.cc:1196] T e8e6af8ae3c94e6d8723112c5b99cd48 P bda28733e0f34538b774a2d723927573: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-2/wal/wals/e8e6af8ae3c94e6d8723112c5b99cd48
I20260619 02:40:10.455435 13592 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43138:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260619 02:40:10.455515 14169 tablet_replica.cc:330] T e8e6af8ae3c94e6d8723112c5b99cd48 P 766e0955e0c54660b574b4b08afd7cff: stopping tablet replica
I20260619 02:40:10.455564 14169 raft_consensus.cc:2229] T e8e6af8ae3c94e6d8723112c5b99cd48 P 766e0955e0c54660b574b4b08afd7cff [term 1 FOLLOWER]: Raft consensus shutting down.
W20260619 02:40:10.455576 13592 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:40:10.455621 14169 raft_consensus.cc:2258] T e8e6af8ae3c94e6d8723112c5b99cd48 P 766e0955e0c54660b574b4b08afd7cff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:40:10.455713 14170 ts_tablet_manager.cc:1939] T e8e6af8ae3c94e6d8723112c5b99cd48 P bda28733e0f34538b774a2d723927573: Deleting consensus metadata
I20260619 02:40:10.455809 14168 ts_tablet_manager.cc:1918] T e8e6af8ae3c94e6d8723112c5b99cd48 P 55daaa9e103e41febd215858550495c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:40:10.455860 14168 log.cc:1196] T e8e6af8ae3c94e6d8723112c5b99cd48 P 55daaa9e103e41febd215858550495c9: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-0/wal/wals/e8e6af8ae3c94e6d8723112c5b99cd48
I20260619 02:40:10.456079 14168 ts_tablet_manager.cc:1939] T e8e6af8ae3c94e6d8723112c5b99cd48 P 55daaa9e103e41febd215858550495c9: Deleting consensus metadata
I20260619 02:40:10.456483 14169 ts_tablet_manager.cc:1905] T e8e6af8ae3c94e6d8723112c5b99cd48 P 766e0955e0c54660b574b4b08afd7cff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:40:10.456534 13579 catalog_manager.cc:4841] TS 55daaa9e103e41febd215858550495c9 (127.5.241.65:46187): tablet e8e6af8ae3c94e6d8723112c5b99cd48 (table test-table [id=f38324ac106e4085800ee6fec0ed9cf8]) successfully deleted
I20260619 02:40:10.456589 13579 catalog_manager.cc:4841] TS bda28733e0f34538b774a2d723927573 (127.5.241.67:36681): tablet e8e6af8ae3c94e6d8723112c5b99cd48 (table test-table [id=f38324ac106e4085800ee6fec0ed9cf8]) successfully deleted
I20260619 02:40:10.456889 13870 tablet_service.cc:1467] Processing CreateTablet for tablet 4e9c2d123a9740ea97bf290f555c17fc (DEFAULT_TABLE table=test-table [id=62a1121291054b219c399e28826eeedc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:40:10.456949 13730 tablet_service.cc:1467] Processing CreateTablet for tablet 4e9c2d123a9740ea97bf290f555c17fc (DEFAULT_TABLE table=test-table [id=62a1121291054b219c399e28826eeedc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:40:10.457015 13870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e9c2d123a9740ea97bf290f555c17fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:10.457039 13730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e9c2d123a9740ea97bf290f555c17fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:10.458071 14169 ts_tablet_manager.cc:1918] T e8e6af8ae3c94e6d8723112c5b99cd48 P 766e0955e0c54660b574b4b08afd7cff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:40:10.458124 14169 log.cc:1196] T e8e6af8ae3c94e6d8723112c5b99cd48 P 766e0955e0c54660b574b4b08afd7cff: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-1/wal/wals/e8e6af8ae3c94e6d8723112c5b99cd48
I20260619 02:40:10.458261 14185 tablet_bootstrap.cc:492] T 4e9c2d123a9740ea97bf290f555c17fc P 55daaa9e103e41febd215858550495c9: Bootstrap starting.
I20260619 02:40:10.458340 14169 ts_tablet_manager.cc:1939] T e8e6af8ae3c94e6d8723112c5b99cd48 P 766e0955e0c54660b574b4b08afd7cff: Deleting consensus metadata
I20260619 02:40:10.458698 14185 tablet_bootstrap.cc:654] T 4e9c2d123a9740ea97bf290f555c17fc P 55daaa9e103e41febd215858550495c9: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:10.458812 13578 catalog_manager.cc:4841] TS 766e0955e0c54660b574b4b08afd7cff (127.5.241.66:36035): tablet e8e6af8ae3c94e6d8723112c5b99cd48 (table test-table [id=f38324ac106e4085800ee6fec0ed9cf8]) successfully deleted
I20260619 02:40:10.459105 14187 tablet_bootstrap.cc:492] T 4e9c2d123a9740ea97bf290f555c17fc P 766e0955e0c54660b574b4b08afd7cff: Bootstrap starting.
I20260619 02:40:10.459378 14187 tablet_bootstrap.cc:654] T 4e9c2d123a9740ea97bf290f555c17fc P 766e0955e0c54660b574b4b08afd7cff: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:10.459794 14010 tablet_service.cc:1467] Processing CreateTablet for tablet 4e9c2d123a9740ea97bf290f555c17fc (DEFAULT_TABLE table=test-table [id=62a1121291054b219c399e28826eeedc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:40:10.459913 14010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e9c2d123a9740ea97bf290f555c17fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:10.459980 14187 tablet_bootstrap.cc:492] T 4e9c2d123a9740ea97bf290f555c17fc P 766e0955e0c54660b574b4b08afd7cff: No bootstrap required, opened a new log
I20260619 02:40:10.460031 14187 ts_tablet_manager.cc:1397] T 4e9c2d123a9740ea97bf290f555c17fc P 766e0955e0c54660b574b4b08afd7cff: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:40:10.460166 14187 raft_consensus.cc:348] T 4e9c2d123a9740ea97bf290f555c17fc P 766e0955e0c54660b574b4b08afd7cff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } } peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } }
I20260619 02:40:10.460256 14187 raft_consensus.cc:374] T 4e9c2d123a9740ea97bf290f555c17fc P 766e0955e0c54660b574b4b08afd7cff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:10.460285 14187 raft_consensus.cc:729] T 4e9c2d123a9740ea97bf290f555c17fc P 766e0955e0c54660b574b4b08afd7cff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 766e0955e0c54660b574b4b08afd7cff, State: Initialized, Role: FOLLOWER
I20260619 02:40:10.460326 14187 consensus_queue.cc:260] T 4e9c2d123a9740ea97bf290f555c17fc P 766e0955e0c54660b574b4b08afd7cff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } } peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } }
I20260619 02:40:10.460423 14187 ts_tablet_manager.cc:1428] T 4e9c2d123a9740ea97bf290f555c17fc P 766e0955e0c54660b574b4b08afd7cff: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 02:40:10.460760 14185 tablet_bootstrap.cc:492] T 4e9c2d123a9740ea97bf290f555c17fc P 55daaa9e103e41febd215858550495c9: No bootstrap required, opened a new log
I20260619 02:40:10.460804 14185 ts_tablet_manager.cc:1397] T 4e9c2d123a9740ea97bf290f555c17fc P 55daaa9e103e41febd215858550495c9: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260619 02:40:10.460968 14185 raft_consensus.cc:348] T 4e9c2d123a9740ea97bf290f555c17fc P 55daaa9e103e41febd215858550495c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } } peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } }
I20260619 02:40:10.461054 14185 raft_consensus.cc:374] T 4e9c2d123a9740ea97bf290f555c17fc P 55daaa9e103e41febd215858550495c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:10.461089 14185 raft_consensus.cc:729] T 4e9c2d123a9740ea97bf290f555c17fc P 55daaa9e103e41febd215858550495c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55daaa9e103e41febd215858550495c9, State: Initialized, Role: FOLLOWER
I20260619 02:40:10.461133 14184 tablet_bootstrap.cc:492] T 4e9c2d123a9740ea97bf290f555c17fc P bda28733e0f34538b774a2d723927573: Bootstrap starting.
I20260619 02:40:10.461131 14185 consensus_queue.cc:260] T 4e9c2d123a9740ea97bf290f555c17fc P 55daaa9e103e41febd215858550495c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } } peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } }
I20260619 02:40:10.461221 14185 ts_tablet_manager.cc:1428] T 4e9c2d123a9740ea97bf290f555c17fc P 55daaa9e103e41febd215858550495c9: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260619 02:40:10.461428 14184 tablet_bootstrap.cc:654] T 4e9c2d123a9740ea97bf290f555c17fc P bda28733e0f34538b774a2d723927573: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:10.462078 14184 tablet_bootstrap.cc:492] T 4e9c2d123a9740ea97bf290f555c17fc P bda28733e0f34538b774a2d723927573: No bootstrap required, opened a new log
I20260619 02:40:10.462157 14184 ts_tablet_manager.cc:1397] T 4e9c2d123a9740ea97bf290f555c17fc P bda28733e0f34538b774a2d723927573: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:40:10.462271 14184 raft_consensus.cc:348] T 4e9c2d123a9740ea97bf290f555c17fc P bda28733e0f34538b774a2d723927573 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } } peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } }
I20260619 02:40:10.462342 14184 raft_consensus.cc:374] T 4e9c2d123a9740ea97bf290f555c17fc P bda28733e0f34538b774a2d723927573 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:10.462373 14184 raft_consensus.cc:729] T 4e9c2d123a9740ea97bf290f555c17fc P bda28733e0f34538b774a2d723927573 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bda28733e0f34538b774a2d723927573, State: Initialized, Role: FOLLOWER
I20260619 02:40:10.462421 14184 consensus_queue.cc:260] T 4e9c2d123a9740ea97bf290f555c17fc P bda28733e0f34538b774a2d723927573 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } } peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } }
I20260619 02:40:10.462510 14184 ts_tablet_manager.cc:1428] T 4e9c2d123a9740ea97bf290f555c17fc P bda28733e0f34538b774a2d723927573: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 02:40:10.511461 14198 raft_consensus.cc:482] T 4e9c2d123a9740ea97bf290f555c17fc P 55daaa9e103e41febd215858550495c9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:40:10.511583 14198 raft_consensus.cc:504] T 4e9c2d123a9740ea97bf290f555c17fc P 55daaa9e103e41febd215858550495c9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } } peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } }
I20260619 02:40:10.511771 14198 leader_election.cc:288] T 4e9c2d123a9740ea97bf290f555c17fc P 55daaa9e103e41febd215858550495c9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 766e0955e0c54660b574b4b08afd7cff (127.5.241.66:36035), bda28733e0f34538b774a2d723927573 (127.5.241.67:36681)
I20260619 02:40:10.512003 13890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e9c2d123a9740ea97bf290f555c17fc" candidate_uuid: "55daaa9e103e41febd215858550495c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "766e0955e0c54660b574b4b08afd7cff" is_pre_election: true
I20260619 02:40:10.512101 13890 raft_consensus.cc:2454] T 4e9c2d123a9740ea97bf290f555c17fc P 766e0955e0c54660b574b4b08afd7cff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55daaa9e103e41febd215858550495c9 in term 0.
I20260619 02:40:10.512147 14030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e9c2d123a9740ea97bf290f555c17fc" candidate_uuid: "55daaa9e103e41febd215858550495c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bda28733e0f34538b774a2d723927573" is_pre_election: true
I20260619 02:40:10.512267 14030 raft_consensus.cc:2454] T 4e9c2d123a9740ea97bf290f555c17fc P bda28733e0f34538b774a2d723927573 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55daaa9e103e41febd215858550495c9 in term 0.
I20260619 02:40:10.512421 13683 leader_election.cc:302] T 4e9c2d123a9740ea97bf290f555c17fc P 55daaa9e103e41febd215858550495c9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55daaa9e103e41febd215858550495c9, bda28733e0f34538b774a2d723927573; no voters:
I20260619 02:40:10.512560 14198 raft_consensus.cc:2781] T 4e9c2d123a9740ea97bf290f555c17fc P 55daaa9e103e41febd215858550495c9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:40:10.512599 14198 raft_consensus.cc:482] T 4e9c2d123a9740ea97bf290f555c17fc P 55daaa9e103e41febd215858550495c9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:40:10.512651 14198 raft_consensus.cc:3037] T 4e9c2d123a9740ea97bf290f555c17fc P 55daaa9e103e41febd215858550495c9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:10.513337 14198 raft_consensus.cc:504] T 4e9c2d123a9740ea97bf290f555c17fc P 55daaa9e103e41febd215858550495c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } } peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } }
I20260619 02:40:10.513537 14198 leader_election.cc:288] T 4e9c2d123a9740ea97bf290f555c17fc P 55daaa9e103e41febd215858550495c9 [CANDIDATE]: Term 1 election: Requested vote from peers 766e0955e0c54660b574b4b08afd7cff (127.5.241.66:36035), bda28733e0f34538b774a2d723927573 (127.5.241.67:36681)
I20260619 02:40:10.513609 14030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e9c2d123a9740ea97bf290f555c17fc" candidate_uuid: "55daaa9e103e41febd215858550495c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bda28733e0f34538b774a2d723927573"
I20260619 02:40:10.513686 14030 raft_consensus.cc:3037] T 4e9c2d123a9740ea97bf290f555c17fc P bda28733e0f34538b774a2d723927573 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:10.513664 13890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e9c2d123a9740ea97bf290f555c17fc" candidate_uuid: "55daaa9e103e41febd215858550495c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "766e0955e0c54660b574b4b08afd7cff"
I20260619 02:40:10.513733 13890 raft_consensus.cc:3037] T 4e9c2d123a9740ea97bf290f555c17fc P 766e0955e0c54660b574b4b08afd7cff [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:10.514437 14030 raft_consensus.cc:2454] T 4e9c2d123a9740ea97bf290f555c17fc P bda28733e0f34538b774a2d723927573 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55daaa9e103e41febd215858550495c9 in term 1.
I20260619 02:40:10.514539 13890 raft_consensus.cc:2454] T 4e9c2d123a9740ea97bf290f555c17fc P 766e0955e0c54660b574b4b08afd7cff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55daaa9e103e41febd215858550495c9 in term 1.
I20260619 02:40:10.514604 13683 leader_election.cc:302] T 4e9c2d123a9740ea97bf290f555c17fc P 55daaa9e103e41febd215858550495c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55daaa9e103e41febd215858550495c9, bda28733e0f34538b774a2d723927573; no voters:
I20260619 02:40:10.514688 14198 raft_consensus.cc:2781] T 4e9c2d123a9740ea97bf290f555c17fc P 55daaa9e103e41febd215858550495c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:40:10.514744 14198 raft_consensus.cc:686] T 4e9c2d123a9740ea97bf290f555c17fc P 55daaa9e103e41febd215858550495c9 [term 1 LEADER]: Becoming Leader. State: Replica: 55daaa9e103e41febd215858550495c9, State: Running, Role: LEADER
I20260619 02:40:10.514818 14198 consensus_queue.cc:237] T 4e9c2d123a9740ea97bf290f555c17fc P 55daaa9e103e41febd215858550495c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } } peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } }
I20260619 02:40:10.515486 13591 catalog_manager.cc:5495] T 4e9c2d123a9740ea97bf290f555c17fc P 55daaa9e103e41febd215858550495c9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 55daaa9e103e41febd215858550495c9 (127.5.241.65). New cstate: current_term: 1 leader_uuid: "55daaa9e103e41febd215858550495c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55daaa9e103e41febd215858550495c9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46187 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 } health_report { overall_health: UNKNOWN } } }
I20260619 02:40:10.528709 14101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:10.525700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:49773 (local address 127.5.241.66:36035)
0619 02:40:10.525744 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:10.525746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:10.525891 (+ 145us) server_negotiation.cc:407] Connection header received
0619 02:40:10.526012 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:10.526013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:10.526024 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:10.526054 (+ 30us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:40:10.526259 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:10.526842 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:10.528072 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:10.528254 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:10.528312 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:10.528509 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:40:10.528542 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:40:10.528578 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:10.528634 (+ 56us) server_negotiation.cc:299] Negotiation successful
0619 02:40:10.528652 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 02:40:10.528780 14206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:10.525653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.66:36035 (local address 127.5.241.65:49773)
0619 02:40:10.525826 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:10.525837 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:10.525925 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:10.526120 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:10.526120 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:10.526122 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:40:10.526199 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:10.526201 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:10.526924 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:10.526925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:10.527962 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:10.527964 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:10.528338 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:10.528339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:10.528410 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:10.528424 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:40:10.528580 (+ 156us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:40:10.528582 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 02:40:10.528632 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 02:40:10.528674 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":98,"threads_started":1}
I20260619 02:40:10.530714 14030 raft_consensus.cc:1261] T 4e9c2d123a9740ea97bf290f555c17fc P bda28733e0f34538b774a2d723927573 [term 1 FOLLOWER]: Refusing update from remote peer 55daaa9e103e41febd215858550495c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:40:10.530783 13890 raft_consensus.cc:1261] T 4e9c2d123a9740ea97bf290f555c17fc P 766e0955e0c54660b574b4b08afd7cff [term 1 FOLLOWER]: Refusing update from remote peer 55daaa9e103e41febd215858550495c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:40:10.530938 14112 consensus_queue.cc:1035] T 4e9c2d123a9740ea97bf290f555c17fc P 55daaa9e103e41febd215858550495c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bda28733e0f34538b774a2d723927573" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:40:10.531041 14198 consensus_queue.cc:1035] T 4e9c2d123a9740ea97bf290f555c17fc P 55daaa9e103e41febd215858550495c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "766e0955e0c54660b574b4b08afd7cff" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:40:10.538929 14195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:10.536327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:35795 (local address 127.5.241.65:46187)
0619 02:40:10.536367 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:10.536369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:10.536492 (+ 123us) server_negotiation.cc:407] Connection header received
0619 02:40:10.536517 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:10.536518 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:10.536529 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:10.536574 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:40:10.536719 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:10.537173 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:10.538208 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:10.538499 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:10.538565 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:10.538703 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:40:10.538727 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:40:10.538757 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:10.538852 (+ 95us) server_negotiation.cc:299] Negotiation successful
0619 02:40:10.538878 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 02:40:10.538983 14165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:10.536324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:46187 (local address 127.5.241.66:35795)
0619 02:40:10.536375 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:10.536386 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:10.536428 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:10.536617 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:10.536618 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:10.536619 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:40:10.536661 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:10.536662 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:10.537261 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:10.537262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:10.538122 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:10.538125 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:10.538567 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:10.538568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:10.538627 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:10.538634 (+ 7us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:40:10.538792 (+ 158us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:40:10.538794 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 02:40:10.538848 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 02:40:10.538896 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260619 02:40:10.547643 13592 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:40:10.547777 13592 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:40:10.548421 13592 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 293499ad3ae14eb6a69f86ebd6da77d9: Sending DeleteTablet for 3 replicas of tablet 4e9c2d123a9740ea97bf290f555c17fc
I20260619 02:40:10.548709 14010 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e9c2d123a9740ea97bf290f555c17fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:40:10 UTC) from {username='oryx', principal='oryx/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:55686
I20260619 02:40:10.548729 13870 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e9c2d123a9740ea97bf290f555c17fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:40:10 UTC) from {username='oryx', principal='oryx/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:41762
I20260619 02:40:10.548733 13730 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e9c2d123a9740ea97bf290f555c17fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:40:10 UTC) from {username='oryx', principal='oryx/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:33812
I20260619 02:40:10.548805 14170 tablet_replica.cc:330] T 4e9c2d123a9740ea97bf290f555c17fc P bda28733e0f34538b774a2d723927573: stopping tablet replica
I20260619 02:40:10.548840 14169 tablet_replica.cc:330] T 4e9c2d123a9740ea97bf290f555c17fc P 766e0955e0c54660b574b4b08afd7cff: stopping tablet replica
I20260619 02:40:10.548858 14170 raft_consensus.cc:2229] T 4e9c2d123a9740ea97bf290f555c17fc P bda28733e0f34538b774a2d723927573 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:40:10.548861 14168 tablet_replica.cc:330] T 4e9c2d123a9740ea97bf290f555c17fc P 55daaa9e103e41febd215858550495c9: stopping tablet replica
I20260619 02:40:10.548895 14170 raft_consensus.cc:2258] T 4e9c2d123a9740ea97bf290f555c17fc P bda28733e0f34538b774a2d723927573 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:40:10.548902 14169 raft_consensus.cc:2229] T 4e9c2d123a9740ea97bf290f555c17fc P 766e0955e0c54660b574b4b08afd7cff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:40:10.548915 14168 raft_consensus.cc:2229] T 4e9c2d123a9740ea97bf290f555c17fc P 55daaa9e103e41febd215858550495c9 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:40:10.548935 14169 raft_consensus.cc:2258] T 4e9c2d123a9740ea97bf290f555c17fc P 766e0955e0c54660b574b4b08afd7cff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:40:10.548972 14168 raft_consensus.cc:2258] T 4e9c2d123a9740ea97bf290f555c17fc P 55daaa9e103e41febd215858550495c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:40:10.549285 14169 ts_tablet_manager.cc:1905] T 4e9c2d123a9740ea97bf290f555c17fc P 766e0955e0c54660b574b4b08afd7cff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:40:10.549301 14170 ts_tablet_manager.cc:1905] T 4e9c2d123a9740ea97bf290f555c17fc P bda28733e0f34538b774a2d723927573: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:40:10.550850 14169 ts_tablet_manager.cc:1918] T 4e9c2d123a9740ea97bf290f555c17fc P 766e0955e0c54660b574b4b08afd7cff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:40:10.550915 14169 log.cc:1196] T 4e9c2d123a9740ea97bf290f555c17fc P 766e0955e0c54660b574b4b08afd7cff: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-1/wal/wals/4e9c2d123a9740ea97bf290f555c17fc
I20260619 02:40:10.550915 14170 ts_tablet_manager.cc:1918] T 4e9c2d123a9740ea97bf290f555c17fc P bda28733e0f34538b774a2d723927573: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:40:10.550981 14170 log.cc:1196] T 4e9c2d123a9740ea97bf290f555c17fc P bda28733e0f34538b774a2d723927573: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836772852275-6085-0/minicluster-data/ts-2/wal/wals/4e9c2d123a9740ea97bf290f555c17fc
I20260619 02:40:10.551146 14169 ts_tablet_manager.cc:1939] T 4e9c2d123a9740ea97bf290f555c17fc P 766e0955e0c54660b574b4b08afd7cff: Deleting consensus metadata
I20260619 02:40:10.551226 14170 ts_tablet_manager.cc:1939] T 4e9c2d123a9740ea97bf290f555c17fc P bda28733e0f34538b774a2d723927573: Deleting consensus metadata
I20260619 02:40:10.551493 13578 catalog_manager.cc:4841] TS 766e0955e0c54660b574b4b08afd7cff (127.5.241.66:36035): tablet 4e9c2d123a9740ea97bf290f555c17fc (table test-table [id=62a1121291054b219c399e28826eeedc]) successfully deleted
I20260619 02:40:10.551677 13579 catalog_manager.cc:4841] TS bda28733e0f34538b774a2d723927573 (127.5.241.67:36681): tablet 4e9c2d123a9740ea97bf290f555c17fc (table test-table [id=62a1121291054b219c399e28826eeedc]) successfully deleted
I20260619 02:40:10.551914 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 13665
W20260619 02:40:10.556802 13579 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.241.65:46187 (error 108)
I20260619 02:40:10.557341 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 13805
I20260619 02:40:10.557361 14210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:10.557177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:46187 (local address 127.0.0.1:33824)
0619 02:40:10.557275 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:10.557320 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.65:46187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81,"thread_start_us":60,"threads_started":1}
W20260619 02:40:10.558266 13579 catalog_manager.cc:4584] TS 55daaa9e103e41febd215858550495c9 (127.5.241.65:46187): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4e9c2d123a9740ea97bf290f555c17fc: Network error: Client connection negotiation failed: client connection to 127.5.241.65:46187: connect: Connection refused (error 111)
I20260619 02:40:10.563359 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 13945
I20260619 02:40:10.568918 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 13561
2026-06-19T02:40:10Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2828 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781836772852275-6085-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:40:10 dist-test-slave-zpbg krb5kdc[14213](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:40:10 dist-test-slave-zpbg krb5kdc[14213](info): set up 2 sockets
Jun 19 02:40:10 dist-test-slave-zpbg krb5kdc[14213](info): commencing operation
krb5kdc: starting...
W20260619 02:40:11.611387 6085 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Jun 19 02:40:11 dist-test-slave-zpbg krb5kdc[14213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836811, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:40:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:40:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.241.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
I20260619 02:40:11.733943 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:42927
--webserver_interface=127.5.241.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:38339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:42927
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:40:11.815589 14226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:11.815797 14226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:11.815819 14226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:11.816880 14226 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:40:11.816916 14226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:11.816931 14226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:11.816943 14226 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:40:11.816955 14226 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:40:11.818470 14226 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:38339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781836772852275-6085-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:42927
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:42927
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.241.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14226
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:40:11.818655 14226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:11.818845 14226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:11.823758 14232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:11.823740 14234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:11.823753 14231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:11.824026 14226 server_base.cc:1034] running on GCE node
I20260619 02:40:11.824294 14226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:11.824576 14226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:11.825749 14226 hybrid_clock.cc:648] HybridClock initialized: now 1781836811825751 us; error 46 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-06-19T02:40:11Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1272 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:40:11 dist-test-slave-zpbg krb5kdc[14239](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:40:11 dist-test-slave-zpbg krb5kdc[14239](info): set up 2 sockets
Jun 19 02:40:11 dist-test-slave-zpbg krb5kdc[14239](info): commencing operation
krb5kdc: starting...
W20260619 02:40:12.887468 6085 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Jun 19 02:40:12 dist-test-slave-zpbg krb5kdc[14239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836812, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:40:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:40:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.241.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
I20260619 02:40:13.006004 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:45565
--webserver_interface=127.5.241.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:34487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:45565
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:40:13.086675 14252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:13.086927 14252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:13.086966 14252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:13.088124 14252 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:40:13.088198 14252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:13.088227 14252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:13.088275 14252 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:40:13.088305 14252 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:40:13.089756 14252 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:34487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:45565
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:45565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.241.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14252
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:40:13.090015 14252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:13.090286 14252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:13.095078 14260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:13.095067 14258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:13.095065 14257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:13.095163 14252 server_base.cc:1034] running on GCE node
I20260619 02:40:13.095424 14252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:13.095682 14252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:13.096845 14252 hybrid_clock.cc:648] HybridClock initialized: now 1781836813096818 us; error 58 us; skew 500 ppm
Jun 19 02:40:13 dist-test-slave-zpbg krb5kdc[14239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836813, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:13.099489 14252 init.cc:377] Logged in from keytab as kudu/127.5.241.126@KRBTEST.COM (short username kudu)
I20260619 02:40:13.099918 14252 webserver.cc:460] Webserver started at http://127.5.241.126:40041/ using document root <none> and password file <none>
I20260619 02:40:13.100183 14252 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:13.100256 14252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:13.100414 14252 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:40:13.101473 14252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/master-0/data/instance:
uuid: "164150cb7c1644b6af9485044e732e10"
format_stamp: "Formatted at 2026-06-19 02:40:13 on dist-test-slave-zpbg"
server_key: "1541f65fc444bb18ff836b8d44047e42"
server_key_iv: "bf655a203de940c574642241d565294a"
server_key_version: "encryptionkey@0"
I20260619 02:40:13.101836 14252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/master-0/wal/instance:
uuid: "164150cb7c1644b6af9485044e732e10"
format_stamp: "Formatted at 2026-06-19 02:40:13 on dist-test-slave-zpbg"
server_key: "1541f65fc444bb18ff836b8d44047e42"
server_key_iv: "bf655a203de940c574642241d565294a"
server_key_version: "encryptionkey@0"
I20260619 02:40:13.103191 14252 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:40:13.103863 14267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:13.104020 14252 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:40:13.104122 14252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "164150cb7c1644b6af9485044e732e10"
format_stamp: "Formatted at 2026-06-19 02:40:13 on dist-test-slave-zpbg"
server_key: "1541f65fc444bb18ff836b8d44047e42"
server_key_iv: "bf655a203de940c574642241d565294a"
server_key_version: "encryptionkey@0"
I20260619 02:40:13.104202 14252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:13.117976 14252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:13.120079 14252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:13.120257 14252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:13.124107 14252 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.126:45565
I20260619 02:40:13.124166 14319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.126:45565 every 8 connection(s)
I20260619 02:40:13.124528 14252 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:40:13.125393 14320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:13.127713 14320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 164150cb7c1644b6af9485044e732e10: Bootstrap starting.
I20260619 02:40:13.128299 14320 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 164150cb7c1644b6af9485044e732e10: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:13.128593 14320 log.cc:824] T 00000000000000000000000000000000 P 164150cb7c1644b6af9485044e732e10: Log is configured to *not* fsync() on all Append() calls
I20260619 02:40:13.129355 14320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 164150cb7c1644b6af9485044e732e10: No bootstrap required, opened a new log
I20260619 02:40:13.130954 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 14252
I20260619 02:40:13.131116 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/master-0/wal/instance
I20260619 02:40:13.130995 14320 raft_consensus.cc:348] T 00000000000000000000000000000000 P 164150cb7c1644b6af9485044e732e10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "164150cb7c1644b6af9485044e732e10" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45565 } }
I20260619 02:40:13.131166 14320 raft_consensus.cc:374] T 00000000000000000000000000000000 P 164150cb7c1644b6af9485044e732e10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:13.131229 14320 raft_consensus.cc:729] T 00000000000000000000000000000000 P 164150cb7c1644b6af9485044e732e10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 164150cb7c1644b6af9485044e732e10, State: Initialized, Role: FOLLOWER
I20260619 02:40:13.131330 6085 external_mini_cluster.cc:1428] Setting key 3f6bdc75ee6e9132d5a941a76e2e5468
I20260619 02:40:13.131404 14320 consensus_queue.cc:260] T 00000000000000000000000000000000 P 164150cb7c1644b6af9485044e732e10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "164150cb7c1644b6af9485044e732e10" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45565 } }
I20260619 02:40:13.131489 14320 raft_consensus.cc:388] T 00000000000000000000000000000000 P 164150cb7c1644b6af9485044e732e10 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:40:13.131548 14320 raft_consensus.cc:482] T 00000000000000000000000000000000 P 164150cb7c1644b6af9485044e732e10 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:40:13.131603 14320 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 164150cb7c1644b6af9485044e732e10 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:13.132684 14320 raft_consensus.cc:504] T 00000000000000000000000000000000 P 164150cb7c1644b6af9485044e732e10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "164150cb7c1644b6af9485044e732e10" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45565 } }
I20260619 02:40:13.132774 14320 leader_election.cc:302] T 00000000000000000000000000000000 P 164150cb7c1644b6af9485044e732e10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 164150cb7c1644b6af9485044e732e10; no voters:
I20260619 02:40:13.132936 14320 leader_election.cc:288] T 00000000000000000000000000000000 P 164150cb7c1644b6af9485044e732e10 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:40:13.133004 14325 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 164150cb7c1644b6af9485044e732e10 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:40:13.133208 14320 sys_catalog.cc:564] T 00000000000000000000000000000000 P 164150cb7c1644b6af9485044e732e10 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:40:13.133231 14325 raft_consensus.cc:686] T 00000000000000000000000000000000 P 164150cb7c1644b6af9485044e732e10 [term 1 LEADER]: Becoming Leader. State: Replica: 164150cb7c1644b6af9485044e732e10, State: Running, Role: LEADER
I20260619 02:40:13.134636 14325 consensus_queue.cc:237] T 00000000000000000000000000000000 P 164150cb7c1644b6af9485044e732e10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "164150cb7c1644b6af9485044e732e10" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45565 } }
I20260619 02:40:13.135263 14326 sys_catalog.cc:455] T 00000000000000000000000000000000 P 164150cb7c1644b6af9485044e732e10 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "164150cb7c1644b6af9485044e732e10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "164150cb7c1644b6af9485044e732e10" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45565 } } }
I20260619 02:40:13.135392 14327 sys_catalog.cc:455] T 00000000000000000000000000000000 P 164150cb7c1644b6af9485044e732e10 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 164150cb7c1644b6af9485044e732e10. Latest consensus state: current_term: 1 leader_uuid: "164150cb7c1644b6af9485044e732e10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "164150cb7c1644b6af9485044e732e10" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45565 } } }
I20260619 02:40:13.136219 14326 sys_catalog.cc:458] T 00000000000000000000000000000000 P 164150cb7c1644b6af9485044e732e10 [sys.catalog]: This master's current role is: LEADER
I20260619 02:40:13.136234 14327 sys_catalog.cc:458] T 00000000000000000000000000000000 P 164150cb7c1644b6af9485044e732e10 [sys.catalog]: This master's current role is: LEADER
W20260619 02:40:13.135691 14339 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 164150cb7c1644b6af9485044e732e10: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:40:13.136456 14339 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:40:13.136703 14342 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:40:13.136981 14342 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 19 02:40:13 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836812, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:40:13.138871 14342 catalog_manager.cc:1347] Generated new cluster ID: ef250b05fa2f46febb44d17df1fb7594
I20260619 02:40:13.138919 14342 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:40:13.142577 14324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:13.131853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52904 (local address 127.5.241.126:45565)
0619 02:40:13.132074 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:13.132088 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:13.132126 (+ 38us) server_negotiation.cc:407] Connection header received
0619 02:40:13.132274 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:13.132284 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:13.132356 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:13.132511 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:13.133341 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:13.134345 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:13.135445 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:13.135899 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:13.135957 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:13.138877 (+ 2920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:13.138880 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:13.138881 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:13.138892 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:13.140979 (+ 2087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:13.141363 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:13.141366 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:13.141366 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:13.141491 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:13.141687 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:13.141688 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:13.141689 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:13.141988 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:13.142164 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:13.142231 (+ 67us) server_negotiation.cc:299] Negotiation successful
0619 02:40:13.142363 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":112,"threads_started":1}
I20260619 02:40:13.146288 14342 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:40:13.146857 14342 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:40:13.151487 14342 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 164150cb7c1644b6af9485044e732e10: Generated new TSK 0
I20260619 02:40:13.151656 14342 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.241.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab.
Entry for principal kudu/127.5.241.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab.
I20260619 02:40:13.177416 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.65:0
--local_ip_for_outbound_sockets=127.5.241.65
--webserver_interface=127.5.241.65
--webserver_port=0
--tserver_master_addrs=127.5.241.126:45565
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab
--principal=kudu/127.5.241.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:34487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:40:13.258298 14346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:13.258507 14346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:13.258531 14346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:13.259771 14346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:13.259814 14346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:13.259841 14346 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.65
I20260619 02:40:13.261300 14346 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:34487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab
--principal=kudu/127.5.241.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.241.65
--webserver_port=0
--tserver_master_addrs=127.5.241.126:45565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14346
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.65
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:40:13.261543 14346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:13.261824 14346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:13.266968 14352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:13.266950 14354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:13.266987 14351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:13.266960 14346 server_base.cc:1034] running on GCE node
I20260619 02:40:13.267344 14346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:13.267601 14346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:13.268770 14346 hybrid_clock.cc:648] HybridClock initialized: now 1781836813268758 us; error 49 us; skew 500 ppm
Jun 19 02:40:13 dist-test-slave-zpbg krb5kdc[14239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836813, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:13.271387 14346 init.cc:377] Logged in from keytab as kudu/127.5.241.65@KRBTEST.COM (short username kudu)
I20260619 02:40:13.271853 14346 webserver.cc:460] Webserver started at http://127.5.241.65:40535/ using document root <none> and password file <none>
I20260619 02:40:13.272092 14346 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:13.272136 14346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:13.272286 14346 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:40:13.273326 14346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-0/data/instance:
uuid: "e9970827620a47c5854387536043d89a"
format_stamp: "Formatted at 2026-06-19 02:40:13 on dist-test-slave-zpbg"
server_key: "6641706bd9cc9409b2b8c2c03cb43d1d"
server_key_iv: "a2a75e4204ab5d6fd9761fdee17965c2"
server_key_version: "encryptionkey@0"
I20260619 02:40:13.273680 14346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-0/wal/instance:
uuid: "e9970827620a47c5854387536043d89a"
format_stamp: "Formatted at 2026-06-19 02:40:13 on dist-test-slave-zpbg"
server_key: "6641706bd9cc9409b2b8c2c03cb43d1d"
server_key_iv: "a2a75e4204ab5d6fd9761fdee17965c2"
server_key_version: "encryptionkey@0"
I20260619 02:40:13.275039 14346 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:40:13.275820 14361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:13.275997 14346 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:40:13.276085 14346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-0/wal
uuid: "e9970827620a47c5854387536043d89a"
format_stamp: "Formatted at 2026-06-19 02:40:13 on dist-test-slave-zpbg"
server_key: "6641706bd9cc9409b2b8c2c03cb43d1d"
server_key_iv: "a2a75e4204ab5d6fd9761fdee17965c2"
server_key_version: "encryptionkey@0"
I20260619 02:40:13.276158 14346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:13.287783 14346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:13.290109 14346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:13.290277 14346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:13.290509 14346 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:40:13.290864 14346 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:40:13.290916 14346 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:13.290961 14346 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:40:13.290987 14346 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:13.296711 14346 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.65:40731
I20260619 02:40:13.296752 14474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.65:40731 every 8 connection(s)
I20260619 02:40:13.297075 14346 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
I20260619 02:40:13.302219 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 14346
I20260619 02:40:13.302315 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-0/wal/instance
I20260619 02:40:13.302510 6085 external_mini_cluster.cc:1428] Setting key 4c6b5a41f3e6be239892e8ea169e1737
Jun 19 02:40:13 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836813, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.65@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:40:13.307005 14324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:13.297888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:51385 (local address 127.5.241.126:45565)
0619 02:40:13.297951 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:13.297953 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:13.298147 (+ 194us) server_negotiation.cc:407] Connection header received
0619 02:40:13.298608 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:13.298609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:13.298624 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:13.298687 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:13.299667 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:13.300191 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:13.301138 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:13.301332 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:13.301391 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:13.303741 (+ 2350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:13.303742 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:13.303743 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:13.303749 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:13.305543 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:13.305948 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:13.305950 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:13.305950 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:13.306014 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:13.306287 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:13.306288 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:13.306288 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:13.306438 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:13.306506 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:13.306752 (+ 246us) server_negotiation.cc:299] Negotiation successful
0619 02:40:13.306870 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 02:40:13.307169 14477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:13.297839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.65:51385)
0619 02:40:13.297991 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:13.298033 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:13.298525 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:13.298805 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:13.298810 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:13.299378 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:13.299565 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:13.299570 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:13.300272 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:13.300272 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:13.301060 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:13.301062 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:13.301409 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:13.301410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:13.301505 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:13.301974 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:13.301982 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:13.303646 (+ 1664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:13.305630 (+ 1984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:13.305636 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:13.305638 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:13.305874 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:13.306101 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:13.306101 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:13.306101 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:13.306202 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:13.306502 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:13.306506 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:13.306622 (+ 116us) client_negotiation.cc:769] Sending connection context
0619 02:40:13.306706 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 02:40:13.306913 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":85,"threads_started":1}
I20260619 02:40:13.307796 14475 heartbeater.cc:347] Connected to a master server at 127.5.241.126:45565
I20260619 02:40:13.307906 14475 heartbeater.cc:464] Registering TS with master...
I20260619 02:40:13.308130 14475 heartbeater.cc:511] Master 127.5.241.126:45565 requested a full tablet report, sending...
I20260619 02:40:13.308651 14284 ts_manager.cc:194] Registered new tserver with Master: e9970827620a47c5854387536043d89a (127.5.241.65:40731)
I20260619 02:40:13.309257 14284 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.65@KRBTEST.COM'} at 127.5.241.65:51385
WARNING: no policy specified for kudu/127.5.241.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab.
Entry for principal kudu/127.5.241.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab.
I20260619 02:40:13.329106 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.66:0
--local_ip_for_outbound_sockets=127.5.241.66
--webserver_interface=127.5.241.66
--webserver_port=0
--tserver_master_addrs=127.5.241.126:45565
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab
--principal=kudu/127.5.241.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:34487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:40:13.412341 14480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:13.412580 14480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:13.412621 14480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:13.413854 14480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:13.413914 14480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:13.413960 14480 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.66
I20260619 02:40:13.415371 14480 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:34487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab
--principal=kudu/127.5.241.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.241.66
--webserver_port=0
--tserver_master_addrs=127.5.241.126:45565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14480
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.66
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:40:13.415627 14480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:13.415845 14480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:40:13.421505 14480 server_base.cc:1034] running on GCE node
W20260619 02:40:13.421501 14486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:13.421804 14485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:13.421810 14488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:13.422055 14480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:13.422318 14480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:13.423487 14480 hybrid_clock.cc:648] HybridClock initialized: now 1781836813423478 us; error 33 us; skew 500 ppm
Jun 19 02:40:13 dist-test-slave-zpbg krb5kdc[14239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836813, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:13.426031 14480 init.cc:377] Logged in from keytab as kudu/127.5.241.66@KRBTEST.COM (short username kudu)
I20260619 02:40:13.426458 14480 webserver.cc:460] Webserver started at http://127.5.241.66:38277/ using document root <none> and password file <none>
I20260619 02:40:13.426684 14480 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:13.426753 14480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:13.426940 14480 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:40:13.428004 14480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-1/data/instance:
uuid: "441d55e8e9c242a08864c35138d265c8"
format_stamp: "Formatted at 2026-06-19 02:40:13 on dist-test-slave-zpbg"
server_key: "c4a43744bc9fa53e4d3e3dd2a908480b"
server_key_iv: "b6b066774b8e1c4eb7b4f07c28de330a"
server_key_version: "encryptionkey@0"
I20260619 02:40:13.428364 14480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-1/wal/instance:
uuid: "441d55e8e9c242a08864c35138d265c8"
format_stamp: "Formatted at 2026-06-19 02:40:13 on dist-test-slave-zpbg"
server_key: "c4a43744bc9fa53e4d3e3dd2a908480b"
server_key_iv: "b6b066774b8e1c4eb7b4f07c28de330a"
server_key_version: "encryptionkey@0"
I20260619 02:40:13.429704 14480 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:40:13.430496 14495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:13.430701 14480 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:40:13.430856 14480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-1/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-1/wal
uuid: "441d55e8e9c242a08864c35138d265c8"
format_stamp: "Formatted at 2026-06-19 02:40:13 on dist-test-slave-zpbg"
server_key: "c4a43744bc9fa53e4d3e3dd2a908480b"
server_key_iv: "b6b066774b8e1c4eb7b4f07c28de330a"
server_key_version: "encryptionkey@0"
I20260619 02:40:13.430938 14480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:13.440783 14480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:13.443019 14480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:13.443192 14480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:13.443440 14480 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:40:13.443782 14480 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:40:13.443838 14480 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:13.443878 14480 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:40:13.443905 14480 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:13.449503 14480 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.66:42651
I20260619 02:40:13.449581 14608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.66:42651 every 8 connection(s)
I20260619 02:40:13.449916 14480 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
I20260619 02:40:13.453917 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 14480
I20260619 02:40:13.454070 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-1/wal/instance
I20260619 02:40:13.454282 6085 external_mini_cluster.cc:1428] Setting key ee8e1d6e96b58f14671417f883226221
Jun 19 02:40:13 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836813, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.66@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:40:13.460505 14324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:13.450815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:56645 (local address 127.5.241.126:45565)
0619 02:40:13.450863 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:13.450867 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:13.450944 (+ 77us) server_negotiation.cc:407] Connection header received
0619 02:40:13.451458 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:13.451460 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:13.451470 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:13.451538 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:13.452539 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:13.453026 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:13.454009 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:13.454351 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:13.454454 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:13.457311 (+ 2857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:13.457313 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:13.457313 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:13.457320 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:13.458979 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:13.459429 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:13.459431 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:13.459432 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:13.459500 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:13.459803 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:13.459804 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:13.459805 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:13.459928 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:13.460014 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:13.460312 (+ 298us) server_negotiation.cc:299] Negotiation successful
0619 02:40:13.460405 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:40:13.460695 14611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:13.450642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.66:56645)
0619 02:40:13.450827 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:13.450870 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:13.451343 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:13.451657 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:13.451661 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:13.452234 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:13.452432 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:13.452436 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:13.453108 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:13.453108 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:13.453915 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:13.453917 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:13.454557 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:13.454559 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:13.454710 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:13.455346 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:13.455363 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:13.457184 (+ 1821us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:13.459080 (+ 1896us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:13.459087 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:13.459089 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:13.459331 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:13.459593 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:13.459594 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:13.459594 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:13.459714 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:13.460002 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:13.460009 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:13.460158 (+ 149us) client_negotiation.cc:769] Sending connection context
0619 02:40:13.460275 (+ 117us) client_negotiation.cc:240] Negotiation successful
0619 02:40:13.460456 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":92,"threads_started":1}
I20260619 02:40:13.461169 14609 heartbeater.cc:347] Connected to a master server at 127.5.241.126:45565
I20260619 02:40:13.461295 14609 heartbeater.cc:464] Registering TS with master...
I20260619 02:40:13.461480 14609 heartbeater.cc:511] Master 127.5.241.126:45565 requested a full tablet report, sending...
I20260619 02:40:13.461925 14284 ts_manager.cc:194] Registered new tserver with Master: 441d55e8e9c242a08864c35138d265c8 (127.5.241.66:42651)
I20260619 02:40:13.462317 14284 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.66@KRBTEST.COM'} at 127.5.241.66:56645
WARNING: no policy specified for kudu/127.5.241.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab.
Entry for principal kudu/127.5.241.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab.
I20260619 02:40:13.478880 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.67:0
--local_ip_for_outbound_sockets=127.5.241.67
--webserver_interface=127.5.241.67
--webserver_port=0
--tserver_master_addrs=127.5.241.126:45565
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab
--principal=kudu/127.5.241.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:34487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:40:13.559054 14614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:13.559254 14614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:13.559284 14614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:13.560478 14614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:13.560521 14614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:13.560547 14614 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.67
I20260619 02:40:13.562000 14614 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:34487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab
--principal=kudu/127.5.241.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.241.67
--webserver_port=0
--tserver_master_addrs=127.5.241.126:45565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.67
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:40:13.562224 14614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:13.562463 14614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:13.567384 14620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:13.567432 14619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:13.567384 14622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:13.567647 14614 server_base.cc:1034] running on GCE node
I20260619 02:40:13.567852 14614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:13.568075 14614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:13.569207 14614 hybrid_clock.cc:648] HybridClock initialized: now 1781836813569178 us; error 58 us; skew 500 ppm
Jun 19 02:40:13 dist-test-slave-zpbg krb5kdc[14239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836813, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:13.571684 14614 init.cc:377] Logged in from keytab as kudu/127.5.241.67@KRBTEST.COM (short username kudu)
I20260619 02:40:13.572139 14614 webserver.cc:460] Webserver started at http://127.5.241.67:43971/ using document root <none> and password file <none>
I20260619 02:40:13.572365 14614 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:13.572432 14614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:13.572582 14614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:40:13.573643 14614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-2/data/instance:
uuid: "440e73f78dc74a1db1bfe52d2290a37d"
format_stamp: "Formatted at 2026-06-19 02:40:13 on dist-test-slave-zpbg"
server_key: "b2e3edd6c6768c325b83da94ef416def"
server_key_iv: "f635e65521844891ea69b8e6bbd5a91a"
server_key_version: "encryptionkey@0"
I20260619 02:40:13.573990 14614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-2/wal/instance:
uuid: "440e73f78dc74a1db1bfe52d2290a37d"
format_stamp: "Formatted at 2026-06-19 02:40:13 on dist-test-slave-zpbg"
server_key: "b2e3edd6c6768c325b83da94ef416def"
server_key_iv: "f635e65521844891ea69b8e6bbd5a91a"
server_key_version: "encryptionkey@0"
I20260619 02:40:13.575356 14614 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260619 02:40:13.576092 14629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:13.576287 14614 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:40:13.576391 14614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-2/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-2/wal
uuid: "440e73f78dc74a1db1bfe52d2290a37d"
format_stamp: "Formatted at 2026-06-19 02:40:13 on dist-test-slave-zpbg"
server_key: "b2e3edd6c6768c325b83da94ef416def"
server_key_iv: "f635e65521844891ea69b8e6bbd5a91a"
server_key_version: "encryptionkey@0"
I20260619 02:40:13.576460 14614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:13.593477 14614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:13.595974 14614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:13.596146 14614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:13.596383 14614 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:40:13.596745 14614 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:40:13.596803 14614 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:13.596843 14614 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:40:13.596871 14614 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:13.602706 14614 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.67:46277
I20260619 02:40:13.602809 14742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.67:46277 every 8 connection(s)
I20260619 02:40:13.603138 14614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
I20260619 02:40:13.603659 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 14614
I20260619 02:40:13.603806 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-2/wal/instance
I20260619 02:40:13.604023 6085 external_mini_cluster.cc:1428] Setting key 98c9c7fcec5ca61871a9f0bec56b47c5
Jun 19 02:40:13 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836813, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.67@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:40:13.613083 14324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:13.604246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:47763 (local address 127.5.241.126:45565)
0619 02:40:13.604304 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:13.604306 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:13.604316 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:40:13.604983 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:13.604984 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:13.604992 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:13.605022 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:13.605942 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:13.606497 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:13.607707 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:13.607911 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:13.608041 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:13.610297 (+ 2256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:13.610299 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:13.610299 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:13.610307 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:13.611683 (+ 1376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:13.612116 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:13.612118 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:13.612118 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:13.612178 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:13.612431 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:13.612432 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:13.612432 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:13.612551 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:13.612629 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:13.612909 (+ 280us) server_negotiation.cc:299] Negotiation successful
0619 02:40:13.612987 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 02:40:13.613308 14745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:13.603953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.67:47763)
0619 02:40:13.604185 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:13.604233 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:13.604888 (+ 655us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:13.605083 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:13.605086 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:13.605651 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:13.605868 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:13.605872 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:13.606593 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:13.606594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:13.607618 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:13.607621 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:13.608054 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:13.608055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:13.608194 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:13.608763 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:13.608771 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:13.610188 (+ 1417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:13.611771 (+ 1583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:13.611792 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:13.611795 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:13.612021 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:13.612248 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:13.612249 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:13.612249 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:13.612358 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:13.612619 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:13.612625 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:13.612767 (+ 142us) client_negotiation.cc:769] Sending connection context
0619 02:40:13.612887 (+ 120us) client_negotiation.cc:240] Negotiation successful
0619 02:40:13.613047 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":93,"threads_started":1}
I20260619 02:40:13.613909 14743 heartbeater.cc:347] Connected to a master server at 127.5.241.126:45565
I20260619 02:40:13.614038 14743 heartbeater.cc:464] Registering TS with master...
I20260619 02:40:13.614249 14743 heartbeater.cc:511] Master 127.5.241.126:45565 requested a full tablet report, sending...
I20260619 02:40:13.614694 14284 ts_manager.cc:194] Registered new tserver with Master: 440e73f78dc74a1db1bfe52d2290a37d (127.5.241.67:46277)
I20260619 02:40:13.615150 14284 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.67@KRBTEST.COM'} at 127.5.241.67:47763
I20260619 02:40:13.615737 6085 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.5.241.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.241.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.241.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.65.keytab.
Entry for principal oryx/127.5.241.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.65.keytab.
WARNING: no policy specified for oryx/127.5.241.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.241.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.241.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.66.keytab.
Entry for principal oryx/127.5.241.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.66.keytab.
WARNING: no policy specified for oryx/127.5.241.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.241.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.241.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.67.keytab.
Entry for principal oryx/127.5.241.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.67.keytab.
I20260619 02:40:13.686748 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 14346
I20260619 02:40:13.691085 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 14480
I20260619 02:40:13.695284 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 14614
I20260619 02:40:13.699391 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 14252
I20260619 02:40:13.704205 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:45565
--webserver_interface=127.5.241.126
--webserver_port=40041
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:34487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:45565
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:40:13.784040 14752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:13.784255 14752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:13.784279 14752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:13.785457 14752 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:40:13.785506 14752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:13.785521 14752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:13.785534 14752 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:40:13.785547 14752 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:40:13.787077 14752 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:34487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:45565
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:45565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.241.126
--webserver_port=40041
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14752
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:40:13.787343 14752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:13.787596 14752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:13.792676 14757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:13.792685 14758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:13.792685 14760 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:13.793052 14752 server_base.cc:1034] running on GCE node
I20260619 02:40:13.793365 14752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:13.793642 14752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:13.794824 14752 hybrid_clock.cc:648] HybridClock initialized: now 1781836813794813 us; error 32 us; skew 500 ppm
Jun 19 02:40:13 dist-test-slave-zpbg krb5kdc[14239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836813, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:13.797475 14752 init.cc:377] Logged in from keytab as kudu/127.5.241.126@KRBTEST.COM (short username kudu)
I20260619 02:40:13.797960 14752 webserver.cc:460] Webserver started at http://127.5.241.126:40041/ using document root <none> and password file <none>
I20260619 02:40:13.798229 14752 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:13.798319 14752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:13.799840 14752 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:40:13.800961 14767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:13.801285 14752 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:40:13.801429 14752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "164150cb7c1644b6af9485044e732e10"
format_stamp: "Formatted at 2026-06-19 02:40:13 on dist-test-slave-zpbg"
server_key: "1541f65fc444bb18ff836b8d44047e42"
server_key_iv: "bf655a203de940c574642241d565294a"
server_key_version: "encryptionkey@0"
I20260619 02:40:13.801834 14752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:13.841884 14752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:13.844489 14752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:13.844674 14752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:13.848944 14752 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.126:45565
I20260619 02:40:13.848980 14819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.126:45565 every 8 connection(s)
I20260619 02:40:13.849378 14752 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:40:13.850967 14820 sys_catalog.cc:263] Verifying existing consensus state
I20260619 02:40:13.851652 14820 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 164150cb7c1644b6af9485044e732e10: Bootstrap starting.
I20260619 02:40:13.859524 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 14752
I20260619 02:40:13.859719 14820 log.cc:824] T 00000000000000000000000000000000 P 164150cb7c1644b6af9485044e732e10: Log is configured to *not* fsync() on all Append() calls
I20260619 02:40:13.859856 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.65:40731
--local_ip_for_outbound_sockets=127.5.241.65
--tserver_master_addrs=127.5.241.126:45565
--webserver_port=40535
--webserver_interface=127.5.241.65
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab
--principal=kudu/127.5.241.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:34487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.5.241.65
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:40:13.861652 14820 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 164150cb7c1644b6af9485044e732e10: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:40:13.861940 14820 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 164150cb7c1644b6af9485044e732e10: Bootstrap complete.
I20260619 02:40:13.865800 14820 raft_consensus.cc:348] T 00000000000000000000000000000000 P 164150cb7c1644b6af9485044e732e10 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "164150cb7c1644b6af9485044e732e10" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45565 } }
I20260619 02:40:13.866192 14820 raft_consensus.cc:729] T 00000000000000000000000000000000 P 164150cb7c1644b6af9485044e732e10 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 164150cb7c1644b6af9485044e732e10, State: Initialized, Role: FOLLOWER
I20260619 02:40:13.866376 14820 consensus_queue.cc:260] T 00000000000000000000000000000000 P 164150cb7c1644b6af9485044e732e10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "164150cb7c1644b6af9485044e732e10" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45565 } }
I20260619 02:40:13.866464 14820 raft_consensus.cc:388] T 00000000000000000000000000000000 P 164150cb7c1644b6af9485044e732e10 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:40:13.866523 14820 raft_consensus.cc:482] T 00000000000000000000000000000000 P 164150cb7c1644b6af9485044e732e10 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:40:13.866582 14820 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 164150cb7c1644b6af9485044e732e10 [term 1 FOLLOWER]: Advancing to term 2
I20260619 02:40:13.868208 14820 raft_consensus.cc:504] T 00000000000000000000000000000000 P 164150cb7c1644b6af9485044e732e10 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "164150cb7c1644b6af9485044e732e10" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45565 } }
I20260619 02:40:13.868335 14820 leader_election.cc:302] T 00000000000000000000000000000000 P 164150cb7c1644b6af9485044e732e10 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 164150cb7c1644b6af9485044e732e10; no voters:
I20260619 02:40:13.868531 14820 leader_election.cc:288] T 00000000000000000000000000000000 P 164150cb7c1644b6af9485044e732e10 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 02:40:13.868607 14824 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 164150cb7c1644b6af9485044e732e10 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 02:40:13.868790 14824 raft_consensus.cc:686] T 00000000000000000000000000000000 P 164150cb7c1644b6af9485044e732e10 [term 2 LEADER]: Becoming Leader. State: Replica: 164150cb7c1644b6af9485044e732e10, State: Running, Role: LEADER
I20260619 02:40:13.868817 14820 sys_catalog.cc:564] T 00000000000000000000000000000000 P 164150cb7c1644b6af9485044e732e10 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:40:13.868884 14824 consensus_queue.cc:237] T 00000000000000000000000000000000 P 164150cb7c1644b6af9485044e732e10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "164150cb7c1644b6af9485044e732e10" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45565 } }
I20260619 02:40:13.869252 14825 sys_catalog.cc:455] T 00000000000000000000000000000000 P 164150cb7c1644b6af9485044e732e10 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "164150cb7c1644b6af9485044e732e10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "164150cb7c1644b6af9485044e732e10" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45565 } } }
I20260619 02:40:13.869325 14825 sys_catalog.cc:458] T 00000000000000000000000000000000 P 164150cb7c1644b6af9485044e732e10 [sys.catalog]: This master's current role is: LEADER
I20260619 02:40:13.869531 14826 sys_catalog.cc:455] T 00000000000000000000000000000000 P 164150cb7c1644b6af9485044e732e10 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 164150cb7c1644b6af9485044e732e10. Latest consensus state: current_term: 2 leader_uuid: "164150cb7c1644b6af9485044e732e10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "164150cb7c1644b6af9485044e732e10" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45565 } } }
I20260619 02:40:13.869658 14826 sys_catalog.cc:458] T 00000000000000000000000000000000 P 164150cb7c1644b6af9485044e732e10 [sys.catalog]: This master's current role is: LEADER
I20260619 02:40:13.869735 14831 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:40:13.870311 14831 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:40:13.870452 14831 catalog_manager.cc:1259] Loaded cluster ID: ef250b05fa2f46febb44d17df1fb7594
I20260619 02:40:13.870687 14831 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:40:13.872120 14831 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:40:13.872293 14831 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 164150cb7c1644b6af9485044e732e10: Loaded TSK: 0
I20260619 02:40:13.872496 14831 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 02:40:13.944121 14823 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:13.944334 14823 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:13.944357 14823 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:13.945536 14823 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:13.945580 14823 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:13.945606 14823 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.65
I20260619 02:40:13.947109 14823 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:34487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.65.keytab
--principal=oryx/127.5.241.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.65:40731
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.241.65
--webserver_port=40535
--tserver_master_addrs=127.5.241.126:45565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14823
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.65
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:40:13.947369 14823 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:13.947609 14823 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:13.953377 14846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:13.953388 14849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:13.953401 14847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:13.953631 14823 server_base.cc:1034] running on GCE node
I20260619 02:40:13.953943 14823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:13.954202 14823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:13.955363 14823 hybrid_clock.cc:648] HybridClock initialized: now 1781836813955354 us; error 36 us; skew 500 ppm
Jun 19 02:40:13 dist-test-slave-zpbg krb5kdc[14239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836813, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.241.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:13.957823 14823 init.cc:377] Logged in from keytab as oryx/127.5.241.65@KRBTEST.COM (short username oryx)
I20260619 02:40:13.958307 14823 webserver.cc:460] Webserver started at http://127.5.241.65:40535/ using document root <none> and password file <none>
I20260619 02:40:13.958550 14823 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:13.958631 14823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:13.960041 14823 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:40:13.960667 14856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:13.960813 14823 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:40:13.960916 14823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-0/wal
uuid: "e9970827620a47c5854387536043d89a"
format_stamp: "Formatted at 2026-06-19 02:40:13 on dist-test-slave-zpbg"
server_key: "6641706bd9cc9409b2b8c2c03cb43d1d"
server_key_iv: "a2a75e4204ab5d6fd9761fdee17965c2"
server_key_version: "encryptionkey@0"
I20260619 02:40:13.961258 14823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:13.983047 14823 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:13.985422 14823 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:13.985571 14823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:13.985874 14823 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:40:13.986258 14823 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:40:13.986296 14823 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:13.986320 14823 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:40:13.986332 14823 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:13.993497 14823 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.65:40731
I20260619 02:40:13.993564 14969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.65:40731 every 8 connection(s)
I20260619 02:40:13.993891 14823 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
I20260619 02:40:13.995452 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 14823
I20260619 02:40:13.995759 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.66:42651
--local_ip_for_outbound_sockets=127.5.241.66
--tserver_master_addrs=127.5.241.126:45565
--webserver_port=38277
--webserver_interface=127.5.241.66
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab
--principal=kudu/127.5.241.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:34487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.5.241.66
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 02:40:13 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.65@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:40:13 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.65@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:40:14.002677 14972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:13.994585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.65:41033)
0619 02:40:13.994746 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:13.994806 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:13.995288 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:13.995775 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:13.995779 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:13.996428 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:13.996612 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:13.996617 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:13.997817 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:13.997818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:13.998665 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:13.998668 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:13.999620 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:13.999621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:13.999746 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:14.000344 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:14.000352 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:14.002526 (+ 2174us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":88,"threads_started":1}
W20260619 02:40:14.002954 14973 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:13.994660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:41033 (local address 127.5.241.126:45565)
0619 02:40:13.994860 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:13.994876 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:13.994903 (+ 27us) server_negotiation.cc:407] Connection header received
0619 02:40:13.995498 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:13.995505 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:13.995574 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:13.995736 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:13.996745 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:13.997715 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:13.998796 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:13.999516 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:13.999647 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:14.002720 (+ 3073us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.241.65:41033: BlockingRecv error: recv got EOF from 127.5.241.65:41033 (error 108)
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":100,"threads_started":1}
W20260619 02:40:14.002966 14970 heartbeater.cc:644] Failed to heartbeat to 127.5.241.126:45565 (0 consecutive failures): Not authorized: Failed to ping master at 127.5.241.126:45565: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
Jun 19 02:40:14 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.65@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:40:14 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.65@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:40:14.008412 14972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:14.003271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.65:48777)
0619 02:40:14.003326 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:14.003337 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:14.003379 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:14.003527 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:14.003528 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:14.003641 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:14.003686 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:14.003687 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:14.004659 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:14.004660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:14.005318 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:14.005320 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:14.005850 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:14.005851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:14.005918 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:14.006421 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:14.006425 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:14.008335 (+ 1910us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":37}
W20260619 02:40:14.008553 14973 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:14.003336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:48777 (local address 127.5.241.126:45565)
0619 02:40:14.003399 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:14.003401 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:14.003410 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:40:14.003434 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:14.003435 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:14.003450 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:14.003521 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:14.003760 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:14.004561 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:14.005402 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:14.005742 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:14.005834 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:14.008482 (+ 2648us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.241.65:48777: BlockingRecv error: recv got EOF from 127.5.241.65:48777 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Jun 19 02:40:14 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.65@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:40:14 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.65@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:40:14.013614 14972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:14.009020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.65:49593)
0619 02:40:14.009093 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:14.009106 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:14.009138 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:14.009308 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:14.009309 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:14.009441 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:14.009470 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:14.009471 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:14.010126 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:14.010127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:14.010622 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:14.010623 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:14.011129 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:14.011130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:14.011201 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:14.011667 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:14.011672 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:14.013544 (+ 1872us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":37}
W20260619 02:40:14.013760 14973 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:14.009059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:49593 (local address 127.5.241.126:45565)
0619 02:40:14.009119 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:14.009121 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:14.009145 (+ 24us) server_negotiation.cc:407] Connection header received
0619 02:40:14.009207 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:14.009208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:14.009220 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:14.009302 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:14.009552 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:14.010006 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:14.010729 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:14.011047 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:14.011129 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:14.013686 (+ 2557us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.241.65:49593: BlockingRecv error: recv got EOF from 127.5.241.65:49593 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260619 02:40:14.013777 14970 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 02:40:14.079072 14974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:14.079275 14974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:14.079305 14974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:14.080521 14974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:14.080561 14974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:14.080587 14974 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.66
I20260619 02:40:14.081930 14974 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:34487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.66.keytab
--principal=oryx/127.5.241.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.66:42651
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.241.66
--webserver_port=38277
--tserver_master_addrs=127.5.241.126:45565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14974
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.66
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:40:14.082111 14974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:14.082278 14974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:14.087237 14982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:14.087239 14979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:14.087289 14980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:14.087328 14974 server_base.cc:1034] running on GCE node
I20260619 02:40:14.087751 14974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:14.088011 14974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:14.089193 14974 hybrid_clock.cc:648] HybridClock initialized: now 1781836814089168 us; error 39 us; skew 500 ppm
Jun 19 02:40:14 dist-test-slave-zpbg krb5kdc[14239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836814, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.241.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:14.091943 14974 init.cc:377] Logged in from keytab as oryx/127.5.241.66@KRBTEST.COM (short username oryx)
I20260619 02:40:14.092392 14974 webserver.cc:460] Webserver started at http://127.5.241.66:38277/ using document root <none> and password file <none>
I20260619 02:40:14.092648 14974 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:14.092741 14974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:14.094066 14974 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:40:14.094651 14989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:14.094888 14974 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:40:14.094985 14974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-1/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-1/wal
uuid: "441d55e8e9c242a08864c35138d265c8"
format_stamp: "Formatted at 2026-06-19 02:40:13 on dist-test-slave-zpbg"
server_key: "c4a43744bc9fa53e4d3e3dd2a908480b"
server_key_iv: "b6b066774b8e1c4eb7b4f07c28de330a"
server_key_version: "encryptionkey@0"
I20260619 02:40:14.095347 14974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:14.131575 14974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:14.133643 14974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:14.133828 14974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:14.134087 14974 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:40:14.134408 14974 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:40:14.134464 14974 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:14.134505 14974 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:40:14.134534 14974 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:14.140290 14974 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.66:42651
I20260619 02:40:14.140341 15102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.66:42651 every 8 connection(s)
I20260619 02:40:14.140700 14974 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
I20260619 02:40:14.140879 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 14974
I20260619 02:40:14.141199 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.67:46277
--local_ip_for_outbound_sockets=127.5.241.67
--tserver_master_addrs=127.5.241.126:45565
--webserver_port=43971
--webserver_interface=127.5.241.67
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab
--principal=kudu/127.5.241.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:34487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.5.241.67
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 02:40:14 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.66@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:40:14 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.66@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
W20260619 02:40:14.149459 14973 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:14.141749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:53549 (local address 127.5.241.126:45565)
0619 02:40:14.141811 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:14.141813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:14.141825 (+ 12us) server_negotiation.cc:407] Connection header received
0619 02:40:14.142547 (+ 722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:14.142549 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:14.142565 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:14.142634 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:14.143691 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:14.144218 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:14.145828 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:14.146013 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:14.146064 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:14.149401 (+ 3337us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.241.66:53549: BlockingRecv error: recv got EOF from 127.5.241.66:53549 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 02:40:14.149524 15105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:14.141453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.66:53549)
0619 02:40:14.141617 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:14.141669 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:14.142426 (+ 757us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:14.142712 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:14.142717 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:14.143365 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:14.143578 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:14.143585 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:14.144316 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:14.144318 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:14.145725 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:14.145728 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:14.146096 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:14.146098 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:14.146257 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:14.146818 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:14.146829 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:14.149327 (+ 2498us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":76,"threads_started":1}
W20260619 02:40:14.149735 15103 heartbeater.cc:644] Failed to heartbeat to 127.5.241.126:45565 (0 consecutive failures): Not authorized: Failed to ping master at 127.5.241.126:45565: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
Jun 19 02:40:14 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.66@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:40:14 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.66@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:40:14.154188 15105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:14.149956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.66:52723)
0619 02:40:14.150009 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:14.150020 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:14.150083 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:14.150201 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:14.150202 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:14.150324 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:14.150364 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:14.150366 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:14.151055 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:14.151056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:14.151552 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:14.151554 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:14.151932 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:14.151932 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:14.152005 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:14.152443 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:14.152447 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:14.154119 (+ 1672us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":36}
W20260619 02:40:14.154268 14973 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:14.150021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:52723 (local address 127.5.241.126:45565)
0619 02:40:14.150085 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:14.150088 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:14.150096 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:40:14.150139 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:14.150140 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:14.150150 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:14.150198 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:14.150452 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:14.150969 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:14.151651 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:14.151849 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:14.151931 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:14.154195 (+ 2264us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.241.66:52723: BlockingRecv error: recv got EOF from 127.5.241.66:52723 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Jun 19 02:40:14 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.66@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:40:14 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.66@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:40:14.158509 15105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:14.154575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.66:41843)
0619 02:40:14.154621 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:14.154631 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:14.154690 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:14.154830 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:14.154831 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:14.154954 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:14.154977 (+ 23us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:14.154979 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:14.155587 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:14.155588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:14.156064 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:14.156065 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:14.156423 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:14.156424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:14.156475 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:14.156892 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:14.156895 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:14.158432 (+ 1537us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":30}
W20260619 02:40:14.158563 14973 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:14.154643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:41843 (local address 127.5.241.126:45565)
0619 02:40:14.154705 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:14.154707 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:14.154715 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:40:14.154737 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:14.154738 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:14.154747 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:14.154826 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:14.155051 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:14.155495 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:14.156158 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:14.156342 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:14.156421 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:14.158496 (+ 2075us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.241.66:41843: BlockingRecv error: recv got EOF from 127.5.241.66:41843 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260619 02:40:14.158833 15103 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 02:40:14.226619 15106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:14.226842 15106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:14.226866 15106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:14.228047 15106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:14.228089 15106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:14.228116 15106 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.67
I20260619 02:40:14.229528 15106 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:34487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/krb5kdc/oryx_127.5.241.67.keytab
--principal=oryx/127.5.241.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.67:46277
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.241.67
--webserver_port=43971
--tserver_master_addrs=127.5.241.126:45565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15106
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.67
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:40:14.229772 15106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:14.229998 15106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:14.235162 15111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:14.235062 15112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:14.235062 15114 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:14.235229 15106 server_base.cc:1034] running on GCE node
I20260619 02:40:14.235598 15106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:14.235877 15106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:14.237041 15106 hybrid_clock.cc:648] HybridClock initialized: now 1781836814237027 us; error 29 us; skew 500 ppm
Jun 19 02:40:14 dist-test-slave-zpbg krb5kdc[14239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836814, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.241.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:14.239539 15106 init.cc:377] Logged in from keytab as oryx/127.5.241.67@KRBTEST.COM (short username oryx)
I20260619 02:40:14.239925 15106 webserver.cc:460] Webserver started at http://127.5.241.67:43971/ using document root <none> and password file <none>
I20260619 02:40:14.240123 15106 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:14.240175 15106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:14.241436 15106 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:40:14.241998 15121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:14.242153 15106 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:40:14.242237 15106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-2/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-2/wal
uuid: "440e73f78dc74a1db1bfe52d2290a37d"
format_stamp: "Formatted at 2026-06-19 02:40:13 on dist-test-slave-zpbg"
server_key: "b2e3edd6c6768c325b83da94ef416def"
server_key_iv: "f635e65521844891ea69b8e6bbd5a91a"
server_key_version: "encryptionkey@0"
I20260619 02:40:14.242550 15106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:14.256403 15106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:14.258639 15106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:14.258750 15106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:14.259012 15106 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:40:14.259318 15106 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:40:14.259351 15106 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:14.259405 15106 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:40:14.259433 15106 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:14.265087 15106 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.67:46277
I20260619 02:40:14.265146 15234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.67:46277 every 8 connection(s)
I20260619 02:40:14.265479 15106 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
I20260619 02:40:14.266489 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 15106
Jun 19 02:40:14 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.67@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:40:14 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.67@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
W20260619 02:40:14.274046 14973 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:14.266219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:49605 (local address 127.5.241.126:45565)
0619 02:40:14.266314 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:14.266317 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:14.266570 (+ 253us) server_negotiation.cc:407] Connection header received
0619 02:40:14.267294 (+ 724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:14.267295 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:14.267305 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:14.267381 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:14.268580 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:14.269172 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:14.270515 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:14.270847 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:14.270910 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:14.273985 (+ 3075us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.241.67:49605: BlockingRecv error: recv got EOF from 127.5.241.67:49605 (error 108)
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":35}
I20260619 02:40:14.274114 15237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:14.266223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.67:49605)
0619 02:40:14.266425 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:14.266474 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:14.267188 (+ 714us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:14.267458 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:14.267462 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:14.268234 (+ 772us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:14.268504 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:14.268509 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:14.269244 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:14.269245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:14.270432 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:14.270435 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:14.270930 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:14.270931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:14.271080 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:14.271509 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:14.271519 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:14.273922 (+ 2403us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":93,"threads_started":1}
W20260619 02:40:14.274334 15235 heartbeater.cc:644] Failed to heartbeat to 127.5.241.126:45565 (0 consecutive failures): Not authorized: Failed to ping master at 127.5.241.126:45565: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
I20260619 02:40:14.275122 15239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:14.266995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52912 (local address 127.5.241.126:45565)
0619 02:40:14.267127 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:14.267129 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:14.267230 (+ 101us) server_negotiation.cc:407] Connection header received
0619 02:40:14.267295 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:14.267295 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:14.267318 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:14.267381 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:14.268173 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:14.268909 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:14.270050 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:14.270249 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:14.270309 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:14.271846 (+ 1537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:14.271850 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:14.271850 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:14.271859 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:14.273585 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:14.273953 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:14.273956 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:14.273957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:14.274052 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:14.274292 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:14.274292 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:14.274293 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:14.274602 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:14.274740 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:14.274894 (+ 154us) server_negotiation.cc:299] Negotiation successful
0619 02:40:14.275007 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":65,"threads_started":1}
Jun 19 02:40:14 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.67@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:40:14 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.67@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:40:14.278560 15237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:14.274551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.67:35013)
0619 02:40:14.274596 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:14.274607 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:14.274646 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:14.274793 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:14.274794 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:14.274922 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:14.274959 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:14.274960 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:14.275723 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:14.275724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:14.276300 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:14.276302 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:14.276671 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:14.276672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:14.276734 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:14.277050 (+ 316us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:14.277053 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:14.278519 (+ 1466us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":31}
W20260619 02:40:14.278715 14973 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:14.274598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:35013 (local address 127.5.241.126:45565)
0619 02:40:14.274637 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:14.274639 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:14.274648 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:40:14.274712 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:14.274713 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:14.274722 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:14.274799 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:14.275036 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:14.275658 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:14.276395 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:14.276591 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:14.276670 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:14.278617 (+ 1947us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.241.67:35013: BlockingRecv error: recv got EOF from 127.5.241.67:35013 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Jun 19 02:40:14 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.67@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:40:14 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.67@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:40:14.282219 15237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:14.278886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.67:43947)
0619 02:40:14.278916 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:14.278926 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:14.278975 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:14.279076 (+ 101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:14.279076 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:14.279173 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:14.279197 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:14.279197 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:14.279743 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:14.279743 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:14.280197 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:14.280198 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:14.280493 (+ 295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:14.280493 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:14.280526 (+ 33us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:14.280833 (+ 307us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:14.280837 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:14.282181 (+ 1344us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":15}
W20260619 02:40:14.282336 14973 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:14.278892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:43947 (local address 127.5.241.126:45565)
0619 02:40:14.278924 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:14.278926 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:14.278957 (+ 31us) server_negotiation.cc:407] Connection header received
0619 02:40:14.279025 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:14.279026 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:14.279033 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:14.279076 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:14.279251 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:14.279663 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:14.280258 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:14.280434 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:14.280480 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:14.282270 (+ 1790us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.241.67:43947: BlockingRecv error: recv got EOF from 127.5.241.67:43947 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260619 02:40:14.282346 15235 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Jun 19 02:40:15 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.65@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:40:15 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.65@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:40:15.019121 15240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:15.014193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.65:35315)
0619 02:40:15.014340 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:15.014356 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:15.014411 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:15.014569 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:15.014570 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:15.014734 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:15.014858 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:15.014860 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:15.015546 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:15.015547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:15.016101 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:15.016103 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:15.016568 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:15.016569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:15.016652 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:15.017154 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:15.017158 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:15.019054 (+ 1896us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":96,"threads_started":1}
W20260619 02:40:15.019304 15241 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:15.014295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:35315 (local address 127.5.241.126:45565)
0619 02:40:15.014429 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:15.014430 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:15.014438 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:40:15.014466 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:15.014467 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:15.014477 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:15.014540 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:15.014932 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:15.015451 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:15.016208 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:15.016409 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:15.016476 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:15.019191 (+ 2715us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.241.65:35315: BlockingRecv error: recv got EOF from 127.5.241.65:35315 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":76,"threads_started":1}
Jun 19 02:40:15 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.66@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:40:15 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.66@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:40:15.164179 15242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:15.159228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.66:47201)
0619 02:40:15.159416 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:15.159427 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:15.159488 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:15.159618 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:15.159619 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:15.159774 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:15.159821 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:15.159822 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:15.160520 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:15.160520 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:15.161376 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:15.161379 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:15.161704 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:15.161705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:15.161740 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:15.162218 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:15.162225 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:15.164059 (+ 1834us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":103,"threads_started":1}
W20260619 02:40:15.164208 15241 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:15.159321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:47201 (local address 127.5.241.126:45565)
0619 02:40:15.159377 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:15.159379 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:15.159498 (+ 119us) server_negotiation.cc:407] Connection header received
0619 02:40:15.159547 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:15.159548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:15.159559 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:15.159604 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:15.159940 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:15.160451 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:15.161453 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:15.161643 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:15.161699 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:15.164162 (+ 2463us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.241.66:47201: BlockingRecv error: recv got EOF from 127.5.241.66:47201 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Jun 19 02:40:15 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.67@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:40:15 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.67@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:40:15.287454 15243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:15.282757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.67:46743)
0619 02:40:15.282887 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:15.282900 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:15.282969 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:15.283087 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:15.283088 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:15.283282 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:15.283329 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:15.283331 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:15.284046 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:15.284047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:15.284619 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:15.284621 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:15.284988 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:15.284989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:15.285047 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:15.285547 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:15.285552 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:15.287335 (+ 1783us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":81,"threads_started":1}
W20260619 02:40:15.287487 15241 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:15.282867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:46743 (local address 127.5.241.126:45565)
0619 02:40:15.282918 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:15.282921 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:15.282950 (+ 29us) server_negotiation.cc:407] Connection header received
0619 02:40:15.283022 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:15.283023 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:15.283036 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:15.283084 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:15.283411 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:15.283951 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:15.284705 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:15.284914 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:15.284991 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:15.287419 (+ 2428us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.241.67:46743: BlockingRecv error: recv got EOF from 127.5.241.67:46743 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Jun 19 02:40:16 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.65@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:40:16 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.65@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:40:16.025578 15244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:16.019639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.65:37997)
0619 02:40:16.019846 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:16.019860 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:16.019945 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:16.020134 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:16.020135 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:16.020373 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:16.020451 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:16.020453 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:16.021426 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:16.021428 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:16.022405 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:16.022409 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:16.022947 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:16.022949 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:16.023019 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:16.023549 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:16.023555 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:16.025519 (+ 1964us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":65,"threads_started":1}
W20260619 02:40:16.025693 15245 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:16.019707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:37997 (local address 127.5.241.126:45565)
0619 02:40:16.019839 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:16.019841 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:16.019923 (+ 82us) server_negotiation.cc:407] Connection header received
0619 02:40:16.020015 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:16.020015 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:16.020046 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:16.020109 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:16.020537 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:16.021333 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:16.022503 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:16.022859 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:16.022931 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:16.025614 (+ 2683us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.241.65:37997: BlockingRecv error: recv got EOF from 127.5.241.65:37997 (error 108)
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":68,"threads_started":1}
Jun 19 02:40:16 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.66@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:40:16 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.66@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:40:16.169407 15246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:16.164674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.66:50547)
0619 02:40:16.164810 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:16.164820 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:16.164868 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:16.165068 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:16.165069 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:16.165248 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:16.165293 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:16.165295 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:16.165989 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:16.165990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:16.166523 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:16.166525 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:16.166953 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:16.166954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:16.167019 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:16.167457 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:16.167462 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:16.169293 (+ 1831us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":79,"threads_started":1}
W20260619 02:40:16.169435 15245 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:16.164882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:50547 (local address 127.5.241.126:45565)
0619 02:40:16.164956 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:16.164958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:16.164964 (+ 6us) server_negotiation.cc:407] Connection header received
0619 02:40:16.164987 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:16.164988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:16.164998 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:16.165060 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:16.165383 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:16.165902 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:16.166610 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:16.166865 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:16.166944 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:16.169375 (+ 2431us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.241.66:50547: BlockingRecv error: recv got EOF from 127.5.241.66:50547 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Jun 19 02:40:16 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.67@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:40:16 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.67@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:40:16.292939 15247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:16.288074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.67:38933)
0619 02:40:16.288243 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:16.288256 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:16.288312 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:16.288463 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:16.288464 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:16.288619 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:16.288669 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:16.288671 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:16.289527 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:16.289528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:16.290170 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:16.290172 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:16.290490 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:16.290491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:16.290533 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:16.290998 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:16.291002 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:16.292881 (+ 1879us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":78,"threads_started":1}
W20260619 02:40:16.293090 15245 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:16.288224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:38933 (local address 127.5.241.126:45565)
0619 02:40:16.288287 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:16.288289 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:16.288333 (+ 44us) server_negotiation.cc:407] Connection header received
0619 02:40:16.288362 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:16.288363 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:16.288375 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:16.288418 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:16.288751 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:16.289427 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:16.290238 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:16.290423 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:16.290477 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:16.293004 (+ 2527us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.241.67:38933: BlockingRecv error: recv got EOF from 127.5.241.67:38933 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Jun 19 02:40:17 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.65@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:40:17 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.65@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:40:17.034723 15248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:17.026117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.65:45541)
0619 02:40:17.026266 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:17.026278 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:17.026337 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:17.026479 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:17.026479 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:17.026651 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:17.026701 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:17.026703 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:17.027449 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:17.027450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:17.027976 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:17.027978 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:17.028357 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:17.028358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:17.028399 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:17.032694 (+ 4295us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:17.032700 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:17.034661 (+ 1961us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":94,"threads_started":1}
W20260619 02:40:17.034945 15249 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:17.026214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:45541 (local address 127.5.241.126:45565)
0619 02:40:17.026362 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:17.026364 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:17.026372 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:40:17.026396 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:17.026396 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:17.026406 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:17.026474 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:17.026836 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:17.027345 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:17.028073 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:17.028276 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:17.028343 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:17.034873 (+ 6530us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.241.65:45541: BlockingRecv error: recv got EOF from 127.5.241.65:45541 (error 108)
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":92,"threads_started":1}
Jun 19 02:40:17 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.66@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:40:17 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.66@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:40:17.174932 15250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:17.169889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.66:45011)
0619 02:40:17.170046 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:17.170060 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:17.170146 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:17.170334 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:17.170335 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:17.170537 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:17.170582 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:17.170584 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:17.171341 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:17.171342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:17.171868 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:17.171869 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:17.172269 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:17.172270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:17.172333 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:17.172793 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:17.172799 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:17.174834 (+ 2035us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":87,"threads_started":1}
W20260619 02:40:17.174985 15249 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:17.170013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:45011 (local address 127.5.241.126:45565)
0619 02:40:17.170076 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:17.170079 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:17.170133 (+ 54us) server_negotiation.cc:407] Connection header received
0619 02:40:17.170237 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:17.170238 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:17.170252 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:17.170342 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:17.170669 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:17.171258 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:17.171965 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:17.172159 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:17.172254 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:17.174918 (+ 2664us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.241.66:45011: BlockingRecv error: recv got EOF from 127.5.241.66:45011 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Jun 19 02:40:17 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.67@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:40:17 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.67@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:40:17.298580 15251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:17.293437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.67:39487)
0619 02:40:17.293614 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:17.293625 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:17.293688 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:17.293831 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:17.293832 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:17.294009 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:17.294057 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:17.294059 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:17.294728 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:17.294729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:17.295566 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:17.295569 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:17.295929 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:17.295930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:17.295971 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:17.296423 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:17.296427 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:17.298484 (+ 2057us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":103,"threads_started":1}
W20260619 02:40:17.298628 15249 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:17.293582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:39487 (local address 127.5.241.126:45565)
0619 02:40:17.293659 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:17.293661 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:17.293675 (+ 14us) server_negotiation.cc:407] Connection header received
0619 02:40:17.293751 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:17.293752 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:17.293764 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:17.293815 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:17.294149 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:17.294651 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:17.295645 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:17.295859 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:17.295916 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:17.298563 (+ 2647us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.241.67:39487: BlockingRecv error: recv got EOF from 127.5.241.67:39487 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Jun 19 02:40:18 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.65@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:40:18 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.65@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:40:18.039923 15252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:18.035276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.65:60765)
0619 02:40:18.035400 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:18.035419 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:18.035527 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:18.035701 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:18.035702 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:18.035884 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:18.035936 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:18.035938 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:18.036588 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:18.036589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:18.037098 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:18.037099 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:18.037474 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:18.037474 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:18.037513 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:18.037979 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:18.037983 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:18.039863 (+ 1880us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":66,"threads_started":1}
W20260619 02:40:18.040015 15253 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:18.035332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:60765 (local address 127.5.241.126:45565)
0619 02:40:18.035442 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:18.035444 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:18.035525 (+ 81us) server_negotiation.cc:407] Connection header received
0619 02:40:18.035602 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:18.035602 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:18.035614 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:18.035677 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:18.036014 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:18.036520 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:18.037179 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:18.037400 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:18.037458 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:18.039963 (+ 2505us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.241.65:60765: BlockingRecv error: recv got EOF from 127.5.241.65:60765 (error 108)
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":55,"threads_started":1}
Jun 19 02:40:18 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.66@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:40:18 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.66@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:40:18.179869 15254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:18.175437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.66:54257)
0619 02:40:18.175595 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:18.175614 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:18.175675 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:18.175828 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:18.175829 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:18.176034 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:18.176082 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:18.176084 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:18.176729 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:18.176730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:18.177273 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:18.177275 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:18.177625 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:18.177626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:18.177676 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:18.178103 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:18.178107 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:18.179811 (+ 1704us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":77,"threads_started":1}
W20260619 02:40:18.180050 15253 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:18.175513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:54257 (local address 127.5.241.126:45565)
0619 02:40:18.175567 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:18.175569 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:18.175684 (+ 115us) server_negotiation.cc:407] Connection header received
0619 02:40:18.175736 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:18.175736 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:18.175749 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:18.175808 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:18.176154 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:18.176637 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:18.177349 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:18.177545 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:18.177596 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:18.179966 (+ 2370us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.241.66:54257: BlockingRecv error: recv got EOF from 127.5.241.66:54257 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Jun 19 02:40:18 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.67@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:40:18 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.67@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:40:18.303705 15255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:18.299064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.67:50895)
0619 02:40:18.299212 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:18.299225 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:18.299287 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:18.299426 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:18.299428 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:18.299587 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:18.299634 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:18.299636 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:18.300337 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:18.300338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:18.300904 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:18.300906 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:18.301285 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:18.301286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:18.301332 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:18.301823 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:18.301827 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:18.303647 (+ 1820us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":103,"threads_started":1}
W20260619 02:40:18.303865 15253 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:18.299128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:50895 (local address 127.5.241.126:45565)
0619 02:40:18.299215 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:18.299216 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:18.299290 (+ 74us) server_negotiation.cc:407] Connection header received
0619 02:40:18.299331 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:18.299332 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:18.299343 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:18.299401 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:18.299726 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:18.300239 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:18.301003 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:18.301208 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:18.301275 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:18.303775 (+ 2500us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.241.67:50895: BlockingRecv error: recv got EOF from 127.5.241.67:50895 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Jun 19 02:40:19 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.65@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:40:19 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.65@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:40:19.045193 15256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:19.040512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.65:39665)
0619 02:40:19.040684 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:19.040698 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:19.040757 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:19.040921 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:19.040922 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:19.041095 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:19.041144 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:19.041146 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:19.041813 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:19.041814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:19.042352 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:19.042354 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:19.042704 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:19.042705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:19.042750 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:19.043218 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:19.043223 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:19.045083 (+ 1860us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":76,"threads_started":1}
W20260619 02:40:19.045245 15257 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:19.040634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:39665 (local address 127.5.241.126:45565)
0619 02:40:19.040736 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:19.040739 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:19.040748 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:40:19.040821 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:19.040822 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:19.040836 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:19.040904 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:19.041218 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:19.041719 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:19.042436 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:19.042633 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:19.042687 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:19.045168 (+ 2481us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.241.65:39665: BlockingRecv error: recv got EOF from 127.5.241.65:39665 (error 108)
Metrics: {"server-negotiator.queue_time_us":80,"thread_start_us":51,"threads_started":1}
Jun 19 02:40:19 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.66@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:40:19 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.66@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:40:19.185601 15258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:19.180390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.66:59019)
0619 02:40:19.180557 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:19.180575 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:19.180628 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:19.180868 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:19.180869 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:19.181064 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:19.181324 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:19.181329 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:19.182105 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:19.182106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:19.182698 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:19.182700 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:19.183111 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:19.183112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:19.183162 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:19.183615 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:19.183620 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:19.185529 (+ 1909us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":61,"threads_started":1}
W20260619 02:40:19.185720 15257 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:19.180619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:59019 (local address 127.5.241.126:45565)
0619 02:40:19.180690 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:19.180692 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:19.180701 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:40:19.180753 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:19.180754 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:19.180766 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:19.180831 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:19.181417 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:19.182020 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:19.182814 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:19.183023 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:19.183088 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:19.185644 (+ 2556us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.241.66:59019: BlockingRecv error: recv got EOF from 127.5.241.66:59019 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Jun 19 02:40:19 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.67@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:40:19 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.67@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:40:19.308830 15259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:19.304251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.67:47855)
0619 02:40:19.304407 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:19.304425 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:19.304480 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:19.304594 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:19.304595 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:19.304765 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:19.304812 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:19.304814 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:19.305578 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:19.305579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:19.306141 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:19.306143 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:19.306474 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:19.306475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:19.306513 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:19.306977 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:19.306981 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:19.308735 (+ 1754us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":108,"threads_started":1}
W20260619 02:40:19.308866 15257 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:19.304310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:47855 (local address 127.5.241.126:45565)
0619 02:40:19.304377 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:19.304379 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:19.304480 (+ 101us) server_negotiation.cc:407] Connection header received
0619 02:40:19.304526 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:19.304527 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:19.304538 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:19.304589 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:19.304935 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:19.305510 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:19.306217 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:19.306407 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:19.306462 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:19.308806 (+ 2344us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.241.67:47855: BlockingRecv error: recv got EOF from 127.5.241.67:47855 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Jun 19 02:40:20 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.65@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:40:20 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.65@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:40:20.050763 15260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:20.045717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.65:39741)
0619 02:40:20.045924 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:20.045934 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:20.045989 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:20.046205 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:20.046205 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:20.046413 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:20.046461 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:20.046462 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:20.047350 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:20.047352 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:20.048028 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:20.048030 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:20.048403 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:20.048404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:20.048443 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:20.048939 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:20.048944 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:20.050695 (+ 1751us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":101,"threads_started":1}
W20260619 02:40:20.050980 15261 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:20.045785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:39741 (local address 127.5.241.126:45565)
0619 02:40:20.045991 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:20.045994 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:20.046049 (+ 55us) server_negotiation.cc:407] Connection header received
0619 02:40:20.046073 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:20.046074 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:20.046084 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:20.046134 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:20.046561 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:20.047239 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:20.048124 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:20.048323 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:20.048406 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:20.050886 (+ 2480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.241.65:39741: BlockingRecv error: recv got EOF from 127.5.241.65:39741 (error 108)
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":137,"threads_started":1}
Jun 19 02:40:20 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.66@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:40:20 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.66@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:40:20.190923 15262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:20.186128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.66:43541)
0619 02:40:20.186268 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:20.186279 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:20.186337 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:20.186485 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:20.186485 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:20.186661 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:20.186707 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:20.186708 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:20.187459 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:20.187460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:20.187997 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:20.187998 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:20.188337 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:20.188338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:20.188384 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:20.188944 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:20.188949 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:20.190866 (+ 1917us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":77,"threads_started":1}
W20260619 02:40:20.191043 15261 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:20.186231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:43541 (local address 127.5.241.126:45565)
0619 02:40:20.186285 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:20.186287 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:20.186323 (+ 36us) server_negotiation.cc:407] Connection header received
0619 02:40:20.186410 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:20.186411 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:20.186422 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:20.186466 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:20.186874 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:20.187375 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:20.188079 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:20.188269 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:20.188331 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:20.190964 (+ 2633us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.241.66:43541: BlockingRecv error: recv got EOF from 127.5.241.66:43541 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Jun 19 02:40:20 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.67@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:40:20 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.67@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:40:20.314386 15263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:20.309396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.67:35037)
0619 02:40:20.309552 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:20.309562 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:20.309619 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:20.309796 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:20.309797 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:20.309991 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:20.310036 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:20.310037 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:20.310757 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:20.310758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:20.311379 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:20.311381 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:20.311749 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:20.311750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:20.311798 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:20.312310 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:20.312316 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:20.314287 (+ 1971us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":93,"threads_started":1}
W20260619 02:40:20.314455 15261 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:20.309450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:35037 (local address 127.5.241.126:45565)
0619 02:40:20.309512 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:20.309514 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:20.309611 (+ 97us) server_negotiation.cc:407] Connection header received
0619 02:40:20.309664 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:20.309664 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:20.309677 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:20.309740 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:20.310135 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:20.310668 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:20.311474 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:20.311665 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:20.311728 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:20.314389 (+ 2661us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.241.67:35037: BlockingRecv error: recv got EOF from 127.5.241.67:35037 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Jun 19 02:40:21 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.65@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:40:21 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.65@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:40:21.056130 15264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:21.051292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.65:36835)
0619 02:40:21.051474 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:21.051485 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:21.051542 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:21.051716 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:21.051718 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:21.051883 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:21.051951 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:21.051952 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:21.052580 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:21.052581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:21.053116 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:21.053118 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:21.053472 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:21.053473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:21.053550 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:21.054089 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:21.054094 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:21.056035 (+ 1941us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":101,"threads_started":1}
W20260619 02:40:21.056174 15265 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:21.051423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:36835 (local address 127.5.241.126:45565)
0619 02:40:21.051556 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:21.051558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:21.051580 (+ 22us) server_negotiation.cc:407] Connection header received
0619 02:40:21.051602 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:21.051602 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:21.051615 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:21.051668 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:21.052029 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:21.052515 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:21.053186 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:21.053372 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:21.053458 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:21.056107 (+ 2649us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.241.65:36835: BlockingRecv error: recv got EOF from 127.5.241.65:36835 (error 108)
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":88,"threads_started":1}
Jun 19 02:40:21 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.66@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:40:21 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.66@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:40:21.196421 15266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:21.191510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.66:57617)
0619 02:40:21.191653 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:21.191670 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:21.191743 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:21.191893 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:21.191894 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:21.192081 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:21.192129 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:21.192131 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:21.192876 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:21.192877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:21.193435 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:21.193437 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:21.193814 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:21.193815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:21.193855 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:21.194455 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:21.194460 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:21.196364 (+ 1904us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":85,"threads_started":1}
W20260619 02:40:21.196548 15265 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:21.191562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:57617 (local address 127.5.241.126:45565)
0619 02:40:21.191647 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:21.191649 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:21.191765 (+ 116us) server_negotiation.cc:407] Connection header received
0619 02:40:21.191798 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:21.191798 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:21.191810 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:21.191880 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:21.192230 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:21.192770 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:21.193536 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:21.193740 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:21.193803 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:21.196470 (+ 2667us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.241.66:57617: BlockingRecv error: recv got EOF from 127.5.241.66:57617 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Jun 19 02:40:21 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.67@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:40:21 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.67@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:40:21.319665 15267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:21.314946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.67:40707)
0619 02:40:21.315132 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:21.315144 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:21.315229 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:21.315399 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:21.315400 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:21.315623 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:21.315685 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:21.315687 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:21.316332 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:21.316333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:21.316869 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:21.316870 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:21.317233 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:21.317234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:21.317279 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:21.317771 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:21.317776 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:21.319608 (+ 1832us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":64,"threads_started":1}
W20260619 02:40:21.319819 15265 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:21.315039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:40707 (local address 127.5.241.126:45565)
0619 02:40:21.315101 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:21.315104 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:21.315225 (+ 121us) server_negotiation.cc:407] Connection header received
0619 02:40:21.315296 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:21.315297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:21.315313 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:21.315390 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:21.315776 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:21.316257 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:21.316957 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:21.317157 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:21.317221 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:21.319729 (+ 2508us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.241.67:40707: BlockingRecv error: recv got EOF from 127.5.241.67:40707 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Jun 19 02:40:22 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.65@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:40:22 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.65@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:40:22.066664 15268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:22.056672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.65:56531)
0619 02:40:22.056881 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:22.056894 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:22.056950 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:22.057128 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:22.057129 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:22.057296 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:22.057348 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:22.057349 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:22.058145 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:22.058146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:22.058689 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:22.058690 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:22.059086 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:22.059087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:22.059130 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:22.064680 (+ 5550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:22.064693 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:22.066594 (+ 1901us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":106,"threads_started":1}
W20260619 02:40:22.066843 15269 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:22.056821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:56531 (local address 127.5.241.126:45565)
0619 02:40:22.056969 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:22.056971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:22.057006 (+ 35us) server_negotiation.cc:407] Connection header received
0619 02:40:22.057029 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:22.057030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:22.057040 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:22.057110 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:22.057432 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:22.058065 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:22.058797 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:22.058990 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:22.059058 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:22.066712 (+ 7654us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.241.65:56531: BlockingRecv error: recv got EOF from 127.5.241.65:56531 (error 108)
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":68,"threads_started":1}
Jun 19 02:40:22 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.66@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:40:22 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.66@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:40:22.202179 15270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:22.196925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.66:43443)
0619 02:40:22.197129 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:22.197146 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:22.197248 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:22.197380 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:22.197380 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:22.197559 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:22.197605 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:22.197607 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:22.198494 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:22.198495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:22.199159 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:22.199161 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:22.199530 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:22.199531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:22.199591 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:22.200038 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:22.200042 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:22.202066 (+ 2024us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":104,"threads_started":1}
W20260619 02:40:22.202210 15269 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:22.197023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:43443 (local address 127.5.241.126:45565)
0619 02:40:22.197102 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:22.197104 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:22.197239 (+ 135us) server_negotiation.cc:407] Connection header received
0619 02:40:22.197308 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:22.197309 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:22.197322 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:22.197378 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:22.197690 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:22.198416 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:22.199242 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:22.199443 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:22.199529 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:22.202147 (+ 2618us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.241.66:43443: BlockingRecv error: recv got EOF from 127.5.241.66:43443 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Jun 19 02:40:22 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.67@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:40:22 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.67@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:40:22.324889 15271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:22.320217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.67:55093)
0619 02:40:22.320375 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:22.320387 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:22.320454 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:22.320580 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:22.320581 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:22.320753 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:22.320803 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:22.320805 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:22.321557 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:22.321558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:22.322161 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:22.322163 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:22.322468 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:22.322469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:22.322506 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:22.323078 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:22.323083 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:22.324831 (+ 1748us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":94,"threads_started":1}
W20260619 02:40:22.325018 15269 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:22.320284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:55093 (local address 127.5.241.126:45565)
0619 02:40:22.320360 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:22.320363 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:22.320456 (+ 93us) server_negotiation.cc:407] Connection header received
0619 02:40:22.320500 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:22.320501 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:22.320512 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:22.320562 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:22.320870 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:22.321480 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:22.322228 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:22.322404 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:22.322457 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:22.324931 (+ 2474us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.241.67:55093: BlockingRecv error: recv got EOF from 127.5.241.67:55093 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Jun 19 02:40:23 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.65@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:40:23 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.65@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:40:23.072000 15272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:23.067211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.65:56331)
0619 02:40:23.067357 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:23.067368 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:23.067422 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:23.067547 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:23.067548 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:23.067733 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:23.067782 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:23.067784 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:23.068480 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:23.068481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:23.069077 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:23.069081 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:23.069471 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:23.069472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:23.069575 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:23.070023 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:23.070028 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:23.071890 (+ 1862us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":94,"threads_started":1}
W20260619 02:40:23.072074 15273 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:23.067277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:56331 (local address 127.5.241.126:45565)
0619 02:40:23.067417 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:23.067418 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:23.067428 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:40:23.067471 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:23.067472 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:23.067481 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:23.067532 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:23.067869 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:23.068399 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:23.069163 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:23.069368 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:23.069423 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:23.071968 (+ 2545us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.241.65:56331: BlockingRecv error: recv got EOF from 127.5.241.65:56331 (error 108)
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":83,"threads_started":1}
Jun 19 02:40:23 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.66@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:40:23 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.66@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:40:23.207563 15274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:23.202636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.66:44477)
0619 02:40:23.202808 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:23.202820 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:23.202891 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:23.203023 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:23.203023 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:23.203233 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:23.203294 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:23.203297 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:23.204203 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:23.204203 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:23.204719 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:23.204720 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:23.205150 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:23.205151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:23.205194 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:23.205664 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:23.205668 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:23.207504 (+ 1836us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":77,"threads_started":1}
W20260619 02:40:23.207676 15273 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:23.202733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:44477 (local address 127.5.241.126:45565)
0619 02:40:23.202830 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:23.202833 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:23.202892 (+ 59us) server_negotiation.cc:407] Connection header received
0619 02:40:23.202948 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:23.202948 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:23.202959 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:23.203002 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:23.203387 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:23.204135 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:23.204819 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:23.205035 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:23.205141 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:23.207599 (+ 2458us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.241.66:44477: BlockingRecv error: recv got EOF from 127.5.241.66:44477 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Jun 19 02:40:23 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.67@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:40:23 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.67@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:40:23.330322 15275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:23.325419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.67:58521)
0619 02:40:23.325579 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:23.325591 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:23.325660 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:23.325812 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:23.325812 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:23.326012 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:23.326055 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:23.326057 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:23.326732 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:23.326734 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:23.327292 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:23.327294 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:23.327671 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:23.327671 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:23.327732 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:23.328210 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:23.328215 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:23.330186 (+ 1971us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":109,"threads_started":1}
W20260619 02:40:23.330328 15273 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:23.325483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:58521 (local address 127.5.241.126:45565)
0619 02:40:23.325546 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:23.325549 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:23.325664 (+ 115us) server_negotiation.cc:407] Connection header received
0619 02:40:23.325714 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:23.325714 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:23.325730 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:23.325805 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:23.326139 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:23.326637 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:23.327392 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:23.327587 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:23.327670 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:23.330268 (+ 2598us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.241.67:58521: BlockingRecv error: recv got EOF from 127.5.241.67:58521 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Jun 19 02:40:24 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.65@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:40:24 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.65@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:40:24.077275 15276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:24.072538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.65:58219)
0619 02:40:24.072719 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:24.072731 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:24.072786 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:24.072943 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:24.072943 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:24.073150 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:24.073200 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:24.073201 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:24.073887 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:24.073889 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:24.074425 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:24.074426 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:24.074875 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:24.074876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:24.074924 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:24.075451 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:24.075456 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:24.077216 (+ 1760us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":99,"threads_started":1}
W20260619 02:40:24.077432 15277 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:24.072640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:58219 (local address 127.5.241.126:45565)
0619 02:40:24.072779 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:24.072781 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:24.072791 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:40:24.072853 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:24.072854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:24.072864 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:24.072923 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:24.073290 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:24.073796 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:24.074526 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:24.074739 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:24.074837 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:24.077344 (+ 2507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.241.65:58219: BlockingRecv error: recv got EOF from 127.5.241.65:58219 (error 108)
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":85,"threads_started":1}
Jun 19 02:40:24 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.66@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:40:24 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.66@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:40:24.213011 15278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:24.208172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.66:39605)
0619 02:40:24.208344 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:24.208357 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:24.208436 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:24.208662 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:24.208663 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:24.208915 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:24.208964 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:24.208966 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:24.209787 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:24.209788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:24.210321 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:24.210323 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:24.210657 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:24.210658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:24.210694 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:24.211184 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:24.211188 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:24.212954 (+ 1766us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":68,"threads_started":1}
W20260619 02:40:24.213148 15277 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:24.208276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:39605 (local address 127.5.241.126:45565)
0619 02:40:24.208351 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:24.208353 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:24.208466 (+ 113us) server_negotiation.cc:407] Connection header received
0619 02:40:24.208500 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:24.208501 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:24.208514 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:24.208637 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:24.209050 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:24.209717 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:24.210401 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:24.210590 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:24.210644 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:24.213054 (+ 2410us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.241.66:39605: BlockingRecv error: recv got EOF from 127.5.241.66:39605 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Jun 19 02:40:24 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.67@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:40:24 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.67@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:40:24.335704 15279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:24.330915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.67:49839)
0619 02:40:24.331082 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:24.331093 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:24.331150 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:24.331322 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:24.331323 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:24.331488 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:24.331537 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:24.331539 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:24.332203 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:24.332204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:24.332744 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:24.332745 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:24.333209 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:24.333210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:24.333256 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:24.333716 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:24.333720 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:24.335639 (+ 1919us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":108,"threads_started":1}
W20260619 02:40:24.335776 15277 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:24.331051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:49839 (local address 127.5.241.126:45565)
0619 02:40:24.331098 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:24.331100 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:24.331136 (+ 36us) server_negotiation.cc:407] Connection header received
0619 02:40:24.331219 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:24.331219 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:24.331229 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:24.331271 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:24.331618 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:24.332111 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:24.332916 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:24.333130 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:24.333199 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:24.335727 (+ 2528us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.241.67:49839: BlockingRecv error: recv got EOF from 127.5.241.67:49839 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Jun 19 02:40:25 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.65@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:40:25 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.65@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:40:25.082425 15280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:25.077792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.65:50903)
0619 02:40:25.077996 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:25.078008 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:25.078095 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:25.078259 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:25.078260 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:25.078443 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:25.078489 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:25.078490 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:25.079138 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:25.079139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:25.079705 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:25.079706 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:25.080074 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:25.080075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:25.080118 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:25.080641 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:25.080646 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:25.082359 (+ 1713us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":121,"threads_started":1}
W20260619 02:40:25.082618 15281 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:25.077911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:50903 (local address 127.5.241.126:45565)
0619 02:40:25.078019 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:25.078021 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:25.078061 (+ 40us) server_negotiation.cc:407] Connection header received
0619 02:40:25.078167 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:25.078168 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:25.078177 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:25.078237 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:25.078555 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:25.079071 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:25.079796 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:25.079993 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:25.080059 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:25.082523 (+ 2464us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.241.65:50903: BlockingRecv error: recv got EOF from 127.5.241.65:50903 (error 108)
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":49,"threads_started":1}
Jun 19 02:40:25 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.66@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:40:25 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.66@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:40:25.218185 15282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:25.213544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.66:49519)
0619 02:40:25.213689 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:25.213707 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:25.213758 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:25.213886 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:25.213887 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:25.214058 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:25.214107 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:25.214114 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:25.214758 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:25.214759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:25.215343 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:25.215345 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:25.215683 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:25.215684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:25.215734 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:25.216233 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:25.216238 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:25.218112 (+ 1874us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":98,"threads_started":1}
W20260619 02:40:25.218299 15281 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:25.213641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:49519 (local address 127.5.241.126:45565)
0619 02:40:25.213720 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:25.213722 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:25.213753 (+ 31us) server_negotiation.cc:407] Connection header received
0619 02:40:25.213806 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:25.213807 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:25.213816 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:25.213871 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:25.214176 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:25.214660 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:25.215419 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:25.215595 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:25.215646 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:25.218202 (+ 2556us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.241.66:49519: BlockingRecv error: recv got EOF from 127.5.241.66:49519 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Jun 19 02:40:25 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.67@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:40:25 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.67@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:40:25.341048 15283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:25.336234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.67:50821)
0619 02:40:25.336363 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:25.336378 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:25.336440 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:25.336634 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:25.336635 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:25.336809 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:25.336855 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:25.336857 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:25.337563 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:25.337564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:25.338130 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:25.338132 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:25.338516 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:25.338517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:25.338571 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:25.339118 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:25.339122 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:25.340934 (+ 1812us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":77,"threads_started":1}
W20260619 02:40:25.341071 15281 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:25.336340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:50821 (local address 127.5.241.126:45565)
0619 02:40:25.336412 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:25.336414 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:25.336423 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:40:25.336544 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:25.336544 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:25.336558 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:25.336628 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:25.336938 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:25.337454 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:25.338218 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:25.338427 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:25.338527 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:25.341008 (+ 2481us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.241.67:50821: BlockingRecv error: recv got EOF from 127.5.241.67:50821 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Jun 19 02:40:26 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.65@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:40:26 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.65@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:40:26.087687 15284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:26.082988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.65:54113)
0619 02:40:26.083130 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:26.083141 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:26.083198 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:26.083317 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:26.083317 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:26.083482 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:26.083530 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:26.083531 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:26.084274 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:26.084275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:26.085003 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:26.085005 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:26.085350 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:26.085351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:26.085391 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:26.085809 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:26.085813 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:26.087629 (+ 1816us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":83,"threads_started":1}
W20260619 02:40:26.087822 15285 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:26.083066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:54113 (local address 127.5.241.126:45565)
0619 02:40:26.083182 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:26.083188 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:26.083198 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:40:26.083244 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:26.083244 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:26.083254 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:26.083306 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:26.083608 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:26.084113 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:26.085065 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:26.085284 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:26.085339 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:26.087734 (+ 2395us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.241.65:54113: BlockingRecv error: recv got EOF from 127.5.241.65:54113 (error 108)
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":63,"threads_started":1}
Jun 19 02:40:26 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.66@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:40:26 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.66@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:40:26.223278 15286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:26.218684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.66:53263)
0619 02:40:26.218836 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:26.218853 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:26.218914 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:26.219044 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:26.219045 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:26.219210 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:26.219261 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:26.219263 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:26.219920 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:26.219921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:26.220467 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:26.220474 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:26.220824 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:26.220824 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:26.220866 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:26.221299 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:26.221303 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:26.223215 (+ 1912us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":115,"threads_started":1}
W20260619 02:40:26.223420 15285 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:26.218743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:53263 (local address 127.5.241.126:45565)
0619 02:40:26.218833 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:26.218835 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:26.218904 (+ 69us) server_negotiation.cc:407] Connection header received
0619 02:40:26.218974 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:26.218974 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:26.218985 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:26.219025 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:26.219325 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:26.219837 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:26.220548 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:26.220746 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:26.220810 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:26.223339 (+ 2529us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.241.66:53263: BlockingRecv error: recv got EOF from 127.5.241.66:53263 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Jun 19 02:40:26 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.67@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:40:26 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.67@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:40:26.346329 15287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:26.341550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.67:33221)
0619 02:40:26.341680 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:26.341692 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:26.341773 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:26.342001 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:26.342002 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:26.342167 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:26.342214 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:26.342215 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:26.343050 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:26.343051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:26.343590 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:26.343591 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:26.343931 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:26.343931 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:26.343970 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:26.344425 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:26.344430 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:26.346274 (+ 1844us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":71,"threads_started":1}
W20260619 02:40:26.346495 15285 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:26.341641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:33221 (local address 127.5.241.126:45565)
0619 02:40:26.341719 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:26.341721 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:26.341747 (+ 26us) server_negotiation.cc:407] Connection header received
0619 02:40:26.341865 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:26.341866 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:26.341879 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:26.341940 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:26.342311 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:26.342884 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:26.343677 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:26.343863 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:26.343920 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:26.346422 (+ 2502us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.241.67:33221: BlockingRecv error: recv got EOF from 127.5.241.67:33221 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Jun 19 02:40:27 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.65@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:40:27 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.65@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:40:27.101183 15288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:27.088125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.65:54355)
0619 02:40:27.088266 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:27.088278 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:27.088335 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:27.088456 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:27.088457 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:27.088618 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:27.088665 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:27.088666 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:27.089412 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:27.089413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:27.089942 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:27.089943 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:27.090361 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:27.090362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:27.090406 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:27.099312 (+ 8906us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:27.099318 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:27.101124 (+ 1806us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":97,"threads_started":1}
W20260619 02:40:27.101351 15289 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:27.088183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:54355 (local address 127.5.241.126:45565)
0619 02:40:27.088322 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:27.088324 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:27.088333 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:40:27.088380 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:27.088381 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:27.088391 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:27.088453 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:27.088760 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:27.089331 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:27.090045 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:27.090278 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:27.090346 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:27.101239 (+ 10893us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.241.65:54355: BlockingRecv error: recv got EOF from 127.5.241.65:54355 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":83,"threads_started":1}
Jun 19 02:40:27 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.66@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:40:27 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.66@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:40:27.228900 15290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:27.223808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.66:46181)
0619 02:40:27.223964 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:27.223991 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:27.224041 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:27.224263 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:27.224264 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:27.224462 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:27.224507 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:27.224509 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:27.225212 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:27.225213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:27.225759 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:27.225761 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:27.226129 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:27.226130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:27.226202 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:27.226688 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:27.226694 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:27.228829 (+ 2135us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":101,"threads_started":1}
W20260619 02:40:27.228969 15289 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:27.224081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:46181 (local address 127.5.241.126:45565)
0619 02:40:27.224128 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:27.224130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:27.224136 (+ 6us) server_negotiation.cc:407] Connection header received
0619 02:40:27.224163 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:27.224163 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:27.224175 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:27.224252 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:27.224596 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:27.225118 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:27.225857 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:27.226053 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:27.226131 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:27.228923 (+ 2792us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.241.66:46181: BlockingRecv error: recv got EOF from 127.5.241.66:46181 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Jun 19 02:40:27 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.67@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:40:27 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.67@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:40:27.351835 15291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:27.346885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.67:52913)
0619 02:40:27.347028 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:27.347039 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:27.347096 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:27.347294 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:27.347294 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:27.347483 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:27.347557 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:27.347560 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:27.348378 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:27.348379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:27.348932 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:27.348934 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:27.349325 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:27.349326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:27.349378 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:27.349873 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:27.349878 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:27.351767 (+ 1889us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":85,"threads_started":1}
W20260619 02:40:27.351994 15289 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:27.346984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:52913 (local address 127.5.241.126:45565)
0619 02:40:27.347057 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:27.347059 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:27.347083 (+ 24us) server_negotiation.cc:407] Connection header received
0619 02:40:27.347141 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:27.347142 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:27.347153 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:27.347266 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:27.347644 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:27.348285 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:27.349027 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:27.349233 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:27.349298 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:27.351896 (+ 2598us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.241.67:52913: BlockingRecv error: recv got EOF from 127.5.241.67:52913 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Jun 19 02:40:28 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.65@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:40:28 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.65@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:40:28.106561 15292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:28.101709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.65:48075)
0619 02:40:28.101871 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:28.101882 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:28.101950 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:28.102091 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:28.102092 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:28.102261 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:28.102306 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:28.102307 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:28.103066 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:28.103067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:28.103617 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:28.103618 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:28.104060 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:28.104061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:28.104166 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:28.104730 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:28.104735 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:28.106494 (+ 1759us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":85,"threads_started":1}
W20260619 02:40:28.106629 15293 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:28.101826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:48075 (local address 127.5.241.126:45565)
0619 02:40:28.101958 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:28.101960 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:28.101990 (+ 30us) server_negotiation.cc:407] Connection header received
0619 02:40:28.102013 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:28.102013 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:28.102023 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:28.102074 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:28.102409 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:28.102992 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:28.103703 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:28.103940 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:28.104070 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:28.106577 (+ 2507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.241.65:48075: BlockingRecv error: recv got EOF from 127.5.241.65:48075 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":75,"threads_started":1}
Jun 19 02:40:28 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.66@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:40:28 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.66@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:40:28.234545 15294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:28.229464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.66:46015)
0619 02:40:28.229677 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:28.229692 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:28.229769 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:28.229946 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:28.229947 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:28.230139 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:28.230183 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:28.230184 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:28.230952 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:28.230953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:28.231493 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:28.231495 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:28.231913 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:28.231914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:28.231970 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:28.232492 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:28.232499 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:28.234445 (+ 1946us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":103,"threads_started":1}
W20260619 02:40:28.234598 15293 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:28.229601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:46015 (local address 127.5.241.126:45565)
0619 02:40:28.229687 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:28.229691 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:28.229747 (+ 56us) server_negotiation.cc:407] Connection header received
0619 02:40:28.229842 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:28.229843 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:28.229855 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:28.229941 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:28.230298 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:28.230865 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:28.231589 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:28.231841 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:28.231914 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:28.234532 (+ 2618us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.241.66:46015: BlockingRecv error: recv got EOF from 127.5.241.66:46015 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Jun 19 02:40:28 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.67@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:40:28 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.67@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:40:28.357271 15295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:28.352360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.67:38989)
0619 02:40:28.352519 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:28.352530 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:28.352581 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:28.352765 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:28.352767 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:28.352935 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:28.352985 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:28.352987 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:28.353738 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:28.353739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:28.354298 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:28.354299 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:28.354636 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:28.354637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:28.354682 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:28.355271 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:28.355277 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:28.357202 (+ 1925us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":94,"threads_started":1}
W20260619 02:40:28.357334 15293 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:28.352488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:38989 (local address 127.5.241.126:45565)
0619 02:40:28.352560 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:28.352562 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:28.352572 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:40:28.352638 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:28.352639 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:28.352651 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:28.352725 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:28.353091 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:28.353641 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:28.354380 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:28.354563 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:28.354618 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:28.357288 (+ 2670us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.241.67:38989: BlockingRecv error: recv got EOF from 127.5.241.67:38989 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Jun 19 02:40:29 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.65@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:40:29 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.65@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:40:29.111948 15296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:29.107198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.65:48265)
0619 02:40:29.107323 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:29.107341 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:29.107389 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:29.107610 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:29.107611 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:29.107804 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:29.107867 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:29.107869 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:29.108639 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:29.108640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:29.109152 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:29.109153 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:29.109506 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:29.109507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:29.109545 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:29.109987 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:29.109992 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:29.111890 (+ 1898us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":80,"threads_started":1}
W20260619 02:40:29.112067 15297 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:29.107321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:48265 (local address 127.5.241.126:45565)
0619 02:40:29.107442 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:29.107444 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:29.107452 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:40:29.107484 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:29.107484 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:29.107494 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:29.107563 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:29.107987 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:29.108570 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:29.109239 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:29.109436 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:29.109499 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:29.111987 (+ 2488us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.241.65:48265: BlockingRecv error: recv got EOF from 127.5.241.65:48265 (error 108)
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":69,"threads_started":1}
Jun 19 02:40:29 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.66@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:40:29 dist-test-slave-zpbg krb5kdc[14239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.241.66@KRBTEST.COM for oryx/127.5.241.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:40:29.240327 15298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:29.235180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45565 (local address 127.5.241.66:51859)
0619 02:40:29.235487 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:29.235500 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:29.235596 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:29.235808 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:29.235809 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:29.235992 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:29.236036 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:29.236038 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:29.236738 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:29.236739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:29.237310 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:29.237312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:29.237685 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:29.237686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:29.237743 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:29.238272 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:29.238277 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:29.240250 (+ 1973us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.241.126:45565: Server oryx/127.5.241.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":134,"threads_started":1}
W20260619 02:40:29.240458 15297 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:29.235285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:51859 (local address 127.5.241.126:45565)
0619 02:40:29.235479 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:29.235481 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:29.235578 (+ 97us) server_negotiation.cc:407] Connection header received
0619 02:40:29.235690 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:29.235691 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:29.235713 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:29.235797 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:29.236122 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:29.236640 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:29.237413 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:29.237609 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:29.237688 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:29.240356 (+ 2668us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.241.66:51859: BlockingRecv error: recv got EOF from 127.5.241.66:51859 (error 108)
Metrics: {"server-negotiator.queue_time_us":159}
I20260619 02:40:29.271494 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 14823
I20260619 02:40:29.276328 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 14974
I20260619 02:40:29.280560 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 15106
I20260619 02:40:29.284722 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 14752
2026-06-19T02:40:29Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17442 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-06-19T02:40:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:40:29Z Disabled control of system clock
I20260619 02:40:29.312413 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:39031
--webserver_interface=127.5.241.126
--webserver_port=0
--builtin_ntp_servers=127.5.241.84:34893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:39031
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260619 02:40:29.393074 15302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:29.393289 15302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:29.393309 15302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:29.394420 15302 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:40:29.394454 15302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:29.394467 15302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:29.394480 15302 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:40:29.394492 15302 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:40:29.395790 15302 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:34893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:39031
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:39031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.241.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15302
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:40:29.396046 15302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:29.396337 15302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:29.401160 15310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:29.401145 15307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:29.401163 15308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:29.401459 15302 server_base.cc:1034] running on GCE node
I20260619 02:40:29.401690 15302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:29.401925 15302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:29.403074 15302 hybrid_clock.cc:648] HybridClock initialized: now 1781836829403059 us; error 28 us; skew 500 ppm
I20260619 02:40:29.404436 15302 webserver.cc:460] Webserver started at http://127.5.241.126:33049/ using document root <none> and password file <none>
I20260619 02:40:29.404683 15302 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:29.404731 15302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:29.404820 15302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:40:29.405813 15302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/master-0/data/instance:
uuid: "5cfb65610806410294ab0e42560d24ff"
format_stamp: "Formatted at 2026-06-19 02:40:29 on dist-test-slave-zpbg"
server_key: "cffeeab64f57a189e378b798279428d3"
server_key_iv: "dbae30956200781edcff5b88c39ebb23"
server_key_version: "encryptionkey@0"
I20260619 02:40:29.406100 15302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/master-0/wal/instance:
uuid: "5cfb65610806410294ab0e42560d24ff"
format_stamp: "Formatted at 2026-06-19 02:40:29 on dist-test-slave-zpbg"
server_key: "cffeeab64f57a189e378b798279428d3"
server_key_iv: "dbae30956200781edcff5b88c39ebb23"
server_key_version: "encryptionkey@0"
I20260619 02:40:29.407505 15302 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:40:29.408320 15316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:29.408636 15302 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:40:29.408720 15302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "5cfb65610806410294ab0e42560d24ff"
format_stamp: "Formatted at 2026-06-19 02:40:29 on dist-test-slave-zpbg"
server_key: "cffeeab64f57a189e378b798279428d3"
server_key_iv: "dbae30956200781edcff5b88c39ebb23"
server_key_version: "encryptionkey@0"
I20260619 02:40:29.408776 15302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:29.425868 15302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:29.426362 15302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:29.426594 15302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:29.431046 15302 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.126:39031
I20260619 02:40:29.431078 15368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.126:39031 every 8 connection(s)
I20260619 02:40:29.431440 15302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:40:29.432245 15369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:29.434940 15369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cfb65610806410294ab0e42560d24ff: Bootstrap starting.
I20260619 02:40:29.435541 15369 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5cfb65610806410294ab0e42560d24ff: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:29.435842 15369 log.cc:824] T 00000000000000000000000000000000 P 5cfb65610806410294ab0e42560d24ff: Log is configured to *not* fsync() on all Append() calls
I20260619 02:40:29.436599 15369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cfb65610806410294ab0e42560d24ff: No bootstrap required, opened a new log
I20260619 02:40:29.436866 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 15302
I20260619 02:40:29.436952 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/master-0/wal/instance
I20260619 02:40:29.437163 6085 external_mini_cluster.cc:1428] Setting key e5d4c09c657d8ba3c9529db20dbe02f9
I20260619 02:40:29.438951 15369 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5cfb65610806410294ab0e42560d24ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cfb65610806410294ab0e42560d24ff" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 39031 } }
I20260619 02:40:29.439117 15369 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5cfb65610806410294ab0e42560d24ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:29.439167 15369 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5cfb65610806410294ab0e42560d24ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cfb65610806410294ab0e42560d24ff, State: Initialized, Role: FOLLOWER
I20260619 02:40:29.439329 15369 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5cfb65610806410294ab0e42560d24ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cfb65610806410294ab0e42560d24ff" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 39031 } }
I20260619 02:40:29.439445 15369 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5cfb65610806410294ab0e42560d24ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:40:29.439489 15369 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5cfb65610806410294ab0e42560d24ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:40:29.439538 15369 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5cfb65610806410294ab0e42560d24ff [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:29.440073 15369 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5cfb65610806410294ab0e42560d24ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cfb65610806410294ab0e42560d24ff" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 39031 } }
I20260619 02:40:29.440191 15369 leader_election.cc:302] T 00000000000000000000000000000000 P 5cfb65610806410294ab0e42560d24ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5cfb65610806410294ab0e42560d24ff; no voters:
I20260619 02:40:29.440407 15369 leader_election.cc:288] T 00000000000000000000000000000000 P 5cfb65610806410294ab0e42560d24ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:40:29.440457 15374 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5cfb65610806410294ab0e42560d24ff [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:40:29.440629 15374 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5cfb65610806410294ab0e42560d24ff [term 1 LEADER]: Becoming Leader. State: Replica: 5cfb65610806410294ab0e42560d24ff, State: Running, Role: LEADER
I20260619 02:40:29.440680 15369 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5cfb65610806410294ab0e42560d24ff [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:40:29.440747 15374 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5cfb65610806410294ab0e42560d24ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cfb65610806410294ab0e42560d24ff" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 39031 } }
I20260619 02:40:29.441114 15374 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cfb65610806410294ab0e42560d24ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5cfb65610806410294ab0e42560d24ff. Latest consensus state: current_term: 1 leader_uuid: "5cfb65610806410294ab0e42560d24ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cfb65610806410294ab0e42560d24ff" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 39031 } } }
I20260619 02:40:29.441355 15374 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cfb65610806410294ab0e42560d24ff [sys.catalog]: This master's current role is: LEADER
I20260619 02:40:29.441269 15375 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cfb65610806410294ab0e42560d24ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5cfb65610806410294ab0e42560d24ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cfb65610806410294ab0e42560d24ff" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 39031 } } }
I20260619 02:40:29.441741 15375 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cfb65610806410294ab0e42560d24ff [sys.catalog]: This master's current role is: LEADER
W20260619 02:40:29.442499 15390 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5cfb65610806410294ab0e42560d24ff: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:40:29.443099 15390 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:40:29.443480 15383 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:40:29.443724 15383 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:40:29.444020 15373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:29.437724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52860 (local address 127.5.241.126:39031)
0619 02:40:29.437926 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:29.437934 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:29.437958 (+ 24us) server_negotiation.cc:407] Connection header received
0619 02:40:29.438399 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:29.438406 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:29.438461 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:29.438572 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:29.438854 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:29.440308 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:29.441541 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:29.442110 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:29.442228 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:29.443143 (+ 915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:29.443149 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:29.443149 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:40:29.443346 (+ 197us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:29.443386 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:40:29.443398 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:29.443457 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:29.443696 (+ 239us) server_negotiation.cc:299] Negotiation successful
0619 02:40:29.443788 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":92,"threads_started":1}
I20260619 02:40:29.445604 15383 catalog_manager.cc:1347] Generated new cluster ID: 80fb5e52e20047d8a6fe53132df79151
I20260619 02:40:29.445659 15383 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:40:29.455404 15383 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:40:29.455984 15383 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:40:29.464908 15383 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5cfb65610806410294ab0e42560d24ff: Generated new TSK 0
I20260619 02:40:29.465103 15383 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:40:29.474514 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.65:0
--local_ip_for_outbound_sockets=127.5.241.65
--webserver_interface=127.5.241.65
--webserver_port=0
--tserver_master_addrs=127.5.241.126:39031
--builtin_ntp_servers=127.5.241.84:34893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260619 02:40:29.558185 15393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:29.558396 15393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:29.558418 15393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:29.559687 15393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:29.559726 15393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:29.559751 15393 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.65
I20260619 02:40:29.561051 15393 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:34893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.241.65
--webserver_port=0
--tserver_master_addrs=127.5.241.126:39031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15393
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.65
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:40:29.561290 15393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:29.561592 15393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:29.567394 15399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:29.567432 15393 server_base.cc:1034] running on GCE node
W20260619 02:40:29.567529 15401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:29.567405 15398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:29.567819 15393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:29.568085 15393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:29.569231 15393 hybrid_clock.cc:648] HybridClock initialized: now 1781836829569219 us; error 26 us; skew 500 ppm
I20260619 02:40:29.570444 15393 webserver.cc:460] Webserver started at http://127.5.241.65:39349/ using document root <none> and password file <none>
I20260619 02:40:29.570693 15393 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:29.570739 15393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:29.570937 15393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:40:29.571959 15393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-0/data/instance:
uuid: "02a8b0e5773c4ee4843e405f74e4923c"
format_stamp: "Formatted at 2026-06-19 02:40:29 on dist-test-slave-zpbg"
server_key: "605b87ce817fd734bd7cc8b0e1558c30"
server_key_iv: "14e1de1e82d2803be82740a3bb21534c"
server_key_version: "encryptionkey@0"
I20260619 02:40:29.572350 15393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-0/wal/instance:
uuid: "02a8b0e5773c4ee4843e405f74e4923c"
format_stamp: "Formatted at 2026-06-19 02:40:29 on dist-test-slave-zpbg"
server_key: "605b87ce817fd734bd7cc8b0e1558c30"
server_key_iv: "14e1de1e82d2803be82740a3bb21534c"
server_key_version: "encryptionkey@0"
I20260619 02:40:29.573601 15393 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260619 02:40:29.574285 15407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:29.574473 15393 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:40:29.574534 15393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-0/wal
uuid: "02a8b0e5773c4ee4843e405f74e4923c"
format_stamp: "Formatted at 2026-06-19 02:40:29 on dist-test-slave-zpbg"
server_key: "605b87ce817fd734bd7cc8b0e1558c30"
server_key_iv: "14e1de1e82d2803be82740a3bb21534c"
server_key_version: "encryptionkey@0"
I20260619 02:40:29.574625 15393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:29.583174 15393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:29.583449 15393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:29.583595 15393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:29.583815 15393 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:40:29.584117 15393 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:40:29.584149 15393 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:29.584170 15393 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:40:29.584218 15393 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:29.589947 15393 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.65:46405
I20260619 02:40:29.589990 15520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.65:46405 every 8 connection(s)
I20260619 02:40:29.590291 15393 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
I20260619 02:40:29.595472 15373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:29.591059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:36061 (local address 127.5.241.126:39031)
0619 02:40:29.591125 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:29.591127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:29.591372 (+ 245us) server_negotiation.cc:407] Connection header received
0619 02:40:29.591900 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:29.591901 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:29.591912 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:29.591964 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:29.592616 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:29.593126 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:29.594092 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:29.594306 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:29.594389 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:29.595166 (+ 777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:29.595167 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:29.595168 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:40:29.595172 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:29.595187 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:40:29.595191 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:29.595244 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:29.595335 (+ 91us) server_negotiation.cc:299] Negotiation successful
0619 02:40:29.595396 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 02:40:29.595654 15523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:29.591038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39031 (local address 127.5.241.65:36061)
0619 02:40:29.591240 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:29.591271 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:29.591771 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:29.592040 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:29.592043 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:29.592055 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:29.592517 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:29.592522 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:29.593214 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:29.593215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:29.594000 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:29.594002 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:29.594397 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:29.594398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:29.594532 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:29.595019 (+ 487us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:40:29.595019 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:29.595045 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:40:29.595066 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:29.595246 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:29.595251 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:29.595254 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:40:29.595337 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 02:40:29.595435 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":84,"threads_started":1}
I20260619 02:40:29.596168 15521 heartbeater.cc:347] Connected to a master server at 127.5.241.126:39031
I20260619 02:40:29.596294 15521 heartbeater.cc:464] Registering TS with master...
I20260619 02:40:29.596482 15521 heartbeater.cc:511] Master 127.5.241.126:39031 requested a full tablet report, sending...
I20260619 02:40:29.596964 15333 ts_manager.cc:194] Registered new tserver with Master: 02a8b0e5773c4ee4843e405f74e4923c (127.5.241.65:46405)
I20260619 02:40:29.597605 15333 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.241.65:36061
I20260619 02:40:29.599555 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 15393
I20260619 02:40:29.599627 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-0/wal/instance
I20260619 02:40:29.599866 6085 external_mini_cluster.cc:1428] Setting key 4a71ade4ab55fd1e9756e29acb7fa61a
I20260619 02:40:29.600754 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.66:0
--local_ip_for_outbound_sockets=127.5.241.66
--webserver_interface=127.5.241.66
--webserver_port=0
--tserver_master_addrs=127.5.241.126:39031
--builtin_ntp_servers=127.5.241.84:34893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260619 02:40:29.685137 15524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:29.685379 15524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:29.685412 15524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:29.686555 15524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:29.686617 15524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:29.686659 15524 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.66
I20260619 02:40:29.688046 15524 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:34893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.241.66
--webserver_port=0
--tserver_master_addrs=127.5.241.126:39031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.66
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:40:29.688313 15524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:29.688609 15524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:29.693590 15529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:29.693616 15530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:29.693619 15532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:29.693910 15524 server_base.cc:1034] running on GCE node
I20260619 02:40:29.694130 15524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:29.694372 15524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:29.695546 15524 hybrid_clock.cc:648] HybridClock initialized: now 1781836829695530 us; error 27 us; skew 500 ppm
I20260619 02:40:29.696918 15524 webserver.cc:460] Webserver started at http://127.5.241.66:38551/ using document root <none> and password file <none>
I20260619 02:40:29.697161 15524 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:29.697235 15524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:29.697402 15524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:40:29.698469 15524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-1/data/instance:
uuid: "d6df877696dc4108b07098699c35f22c"
format_stamp: "Formatted at 2026-06-19 02:40:29 on dist-test-slave-zpbg"
server_key: "412850ea04221355226dfcda376306f3"
server_key_iv: "51184597e824d1ae3311e9919d656ef7"
server_key_version: "encryptionkey@0"
I20260619 02:40:29.698902 15524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-1/wal/instance:
uuid: "d6df877696dc4108b07098699c35f22c"
format_stamp: "Formatted at 2026-06-19 02:40:29 on dist-test-slave-zpbg"
server_key: "412850ea04221355226dfcda376306f3"
server_key_iv: "51184597e824d1ae3311e9919d656ef7"
server_key_version: "encryptionkey@0"
I20260619 02:40:29.700194 15524 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:40:29.700886 15538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:29.701061 15524 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:40:29.701159 15524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-1/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-1/wal
uuid: "d6df877696dc4108b07098699c35f22c"
format_stamp: "Formatted at 2026-06-19 02:40:29 on dist-test-slave-zpbg"
server_key: "412850ea04221355226dfcda376306f3"
server_key_iv: "51184597e824d1ae3311e9919d656ef7"
server_key_version: "encryptionkey@0"
I20260619 02:40:29.701246 15524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:29.723837 15524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:29.724186 15524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:29.724359 15524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:29.724597 15524 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:40:29.724946 15524 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:40:29.725004 15524 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:29.725046 15524 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:40:29.725075 15524 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:29.730876 15524 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.66:41495
I20260619 02:40:29.730954 15651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.66:41495 every 8 connection(s)
I20260619 02:40:29.731310 15524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
I20260619 02:40:29.735930 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 15524
I20260619 02:40:29.736047 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-1/wal/instance
I20260619 02:40:29.736316 6085 external_mini_cluster.cc:1428] Setting key 6b027ac02e08397f0847d6f01d492cd9
I20260619 02:40:29.736872 15373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:29.732073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:51843 (local address 127.5.241.126:39031)
0619 02:40:29.732144 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:29.732146 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:29.732374 (+ 228us) server_negotiation.cc:407] Connection header received
0619 02:40:29.732865 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:29.732866 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:29.732875 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:29.732955 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:29.733719 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:29.734242 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:29.735314 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:29.735494 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:29.735550 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:29.736505 (+ 955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:29.736507 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:29.736507 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:40:29.736513 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:29.736532 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:40:29.736537 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:29.736599 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:29.736766 (+ 167us) server_negotiation.cc:299] Negotiation successful
0619 02:40:29.736791 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 02:40:29.737166 15654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:29.732042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39031 (local address 127.5.241.66:51843)
0619 02:40:29.732245 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:29.732287 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:29.732783 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:29.733059 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:29.733063 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:29.733072 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:29.733586 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:29.733592 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:29.734334 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:29.734335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:29.735235 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:29.735238 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:29.735577 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:29.735578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:29.735687 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:29.736331 (+ 644us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:40:29.736332 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:29.736371 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:40:29.736400 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:29.736684 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:29.736690 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:29.736693 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:40:29.736765 (+ 72us) client_negotiation.cc:240] Negotiation successful
0619 02:40:29.736844 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":109,"threads_started":1}
I20260619 02:40:29.737459 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.67:0
--local_ip_for_outbound_sockets=127.5.241.67
--webserver_interface=127.5.241.67
--webserver_port=0
--tserver_master_addrs=127.5.241.126:39031
--builtin_ntp_servers=127.5.241.84:34893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260619 02:40:29.737607 15652 heartbeater.cc:347] Connected to a master server at 127.5.241.126:39031
I20260619 02:40:29.737703 15652 heartbeater.cc:464] Registering TS with master...
I20260619 02:40:29.737883 15652 heartbeater.cc:511] Master 127.5.241.126:39031 requested a full tablet report, sending...
I20260619 02:40:29.738265 15333 ts_manager.cc:194] Registered new tserver with Master: d6df877696dc4108b07098699c35f22c (127.5.241.66:41495)
I20260619 02:40:29.738790 15333 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.241.66:51843
W20260619 02:40:29.818281 15655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:29.818485 15655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:29.818506 15655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:29.819674 15655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:29.819718 15655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:29.819743 15655 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.67
I20260619 02:40:29.821095 15655 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:34893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.241.67
--webserver_port=0
--tserver_master_addrs=127.5.241.126:39031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15655
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.67
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:40:29.821349 15655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:29.821671 15655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:40:29.826647 15655 server_base.cc:1034] running on GCE node
W20260619 02:40:29.826601 15660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:29.826601 15663 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:29.826617 15661 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:29.827015 15655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:29.827281 15655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:29.828423 15655 hybrid_clock.cc:648] HybridClock initialized: now 1781836829828409 us; error 37 us; skew 500 ppm
I20260619 02:40:29.829614 15655 webserver.cc:460] Webserver started at http://127.5.241.67:40849/ using document root <none> and password file <none>
I20260619 02:40:29.829860 15655 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:29.829907 15655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:29.830072 15655 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:40:29.831254 15655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-2/data/instance:
uuid: "5ff1d1a82e1947a086147279d6c2b466"
format_stamp: "Formatted at 2026-06-19 02:40:29 on dist-test-slave-zpbg"
server_key: "8e9db00152c4dd5d58529f3ad941ba2f"
server_key_iv: "61e3905344a0e3136249e21639a012ba"
server_key_version: "encryptionkey@0"
I20260619 02:40:29.831633 15655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-2/wal/instance:
uuid: "5ff1d1a82e1947a086147279d6c2b466"
format_stamp: "Formatted at 2026-06-19 02:40:29 on dist-test-slave-zpbg"
server_key: "8e9db00152c4dd5d58529f3ad941ba2f"
server_key_iv: "61e3905344a0e3136249e21639a012ba"
server_key_version: "encryptionkey@0"
I20260619 02:40:29.832991 15655 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:40:29.833781 15669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:29.833976 15655 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:40:29.834096 15655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-2/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-2/wal
uuid: "5ff1d1a82e1947a086147279d6c2b466"
format_stamp: "Formatted at 2026-06-19 02:40:29 on dist-test-slave-zpbg"
server_key: "8e9db00152c4dd5d58529f3ad941ba2f"
server_key_iv: "61e3905344a0e3136249e21639a012ba"
server_key_version: "encryptionkey@0"
I20260619 02:40:29.834172 15655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:29.845168 15655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:29.845479 15655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:29.845630 15655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:29.845873 15655 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:40:29.846197 15655 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:40:29.846252 15655 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:29.846307 15655 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:40:29.846331 15655 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:29.852167 15655 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.67:36957
I20260619 02:40:29.852210 15782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.67:36957 every 8 connection(s)
I20260619 02:40:29.852537 15655 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
I20260619 02:40:29.857592 15373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:29.853259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:43601 (local address 127.5.241.126:39031)
0619 02:40:29.853304 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:29.853306 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:29.853554 (+ 248us) server_negotiation.cc:407] Connection header received
0619 02:40:29.854018 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:29.854019 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:29.854029 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:29.854079 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:29.854700 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:29.855238 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:29.856334 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:29.856521 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:29.856569 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:29.857391 (+ 822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:29.857392 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:29.857392 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:40:29.857396 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:29.857409 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:40:29.857412 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:29.857453 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:29.857511 (+ 58us) server_negotiation.cc:299] Negotiation successful
0619 02:40:29.857532 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 02:40:29.857801 15785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:29.853262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39031 (local address 127.5.241.67:43601)
0619 02:40:29.853433 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:29.853466 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:29.853938 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:29.854131 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:29.854134 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:29.854142 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:29.854635 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:29.854639 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:29.855345 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:29.855346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:29.856247 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:29.856250 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:29.856609 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:29.856610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:29.856731 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:29.857251 (+ 520us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:40:29.857251 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:29.857279 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:40:29.857302 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:29.857453 (+ 151us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:29.857455 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:29.857458 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:40:29.857505 (+ 47us) client_negotiation.cc:240] Negotiation successful
0619 02:40:29.857580 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":87,"threads_started":1}
I20260619 02:40:29.858287 15783 heartbeater.cc:347] Connected to a master server at 127.5.241.126:39031
I20260619 02:40:29.858389 15783 heartbeater.cc:464] Registering TS with master...
I20260619 02:40:29.858590 15783 heartbeater.cc:511] Master 127.5.241.126:39031 requested a full tablet report, sending...
I20260619 02:40:29.858994 15333 ts_manager.cc:194] Registered new tserver with Master: 5ff1d1a82e1947a086147279d6c2b466 (127.5.241.67:36957)
I20260619 02:40:29.859308 15333 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.241.67:43601
I20260619 02:40:29.862536 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 15655
I20260619 02:40:29.862622 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-2/wal/instance
I20260619 02:40:29.862872 6085 external_mini_cluster.cc:1428] Setting key a4b79a2b78eef7777278b510f36b9005
I20260619 02:40:29.863737 6085 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 02:40:29.865517 15373 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:29.864988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52876 (local address 127.5.241.126:39031)
0619 02:40:29.865059 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:29.865061 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:29.865155 (+ 94us) server_negotiation.cc:407] Connection header received
0619 02:40:29.865228 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:29.865229 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:29.865240 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:29.865289 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:29.865426 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52876: BlockingRecv error: recv got EOF from 127.0.0.1:52876 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 02:40:29.865597 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 15393
I20260619 02:40:29.870170 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 15524
I20260619 02:40:29.874704 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 15655
I20260619 02:40:29.879035 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 15302
2026-06-19T02:40:29Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (593 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-06-19T02:40:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:40:29Z Disabled control of system clock
I20260619 02:40:29.905746 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:43953
--webserver_interface=127.5.241.126
--webserver_port=0
--builtin_ntp_servers=127.5.241.84:42241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:43953
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 02:40:29.990136 15794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:29.990351 15794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:29.990373 15794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:29.991564 15794 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:40:29.991607 15794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:29.991621 15794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:29.991633 15794 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:40:29.991691 15794 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:40:29.993038 15794 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:42241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:43953
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:43953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.241.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15794
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:40:29.993304 15794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:29.993597 15794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:40:29.998454 15794 server_base.cc:1034] running on GCE node
W20260619 02:40:29.998427 15802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:29.998420 15800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:29.998420 15799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:29.998929 15794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:29.999217 15794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:30.000365 15794 hybrid_clock.cc:648] HybridClock initialized: now 1781836830000329 us; error 43 us; skew 500 ppm
I20260619 02:40:30.001636 15794 webserver.cc:460] Webserver started at http://127.5.241.126:42315/ using document root <none> and password file <none>
I20260619 02:40:30.001892 15794 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:30.001941 15794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:30.002072 15794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:40:30.003161 15794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/master-0/data/instance:
uuid: "7d6cad4dbc39438196d83614134b4e90"
format_stamp: "Formatted at 2026-06-19 02:40:29 on dist-test-slave-zpbg"
server_key: "67cb88eae9d98dd8748ae3ceb7551827"
server_key_iv: "62796952cb6a40a30bf6b5a9330447a6"
server_key_version: "encryptionkey@0"
I20260619 02:40:30.003504 15794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/master-0/wal/instance:
uuid: "7d6cad4dbc39438196d83614134b4e90"
format_stamp: "Formatted at 2026-06-19 02:40:29 on dist-test-slave-zpbg"
server_key: "67cb88eae9d98dd8748ae3ceb7551827"
server_key_iv: "62796952cb6a40a30bf6b5a9330447a6"
server_key_version: "encryptionkey@0"
I20260619 02:40:30.004796 15794 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:40:30.005576 15808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:30.006102 15794 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:40:30.006186 15794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "7d6cad4dbc39438196d83614134b4e90"
format_stamp: "Formatted at 2026-06-19 02:40:29 on dist-test-slave-zpbg"
server_key: "67cb88eae9d98dd8748ae3ceb7551827"
server_key_iv: "62796952cb6a40a30bf6b5a9330447a6"
server_key_version: "encryptionkey@0"
I20260619 02:40:30.006315 15794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:30.009670 15794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:30.009948 15794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:30.010107 15794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:30.013912 15794 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.126:43953
I20260619 02:40:30.013947 15860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.126:43953 every 8 connection(s)
I20260619 02:40:30.014293 15794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:40:30.015012 15861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:30.017735 15861 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d6cad4dbc39438196d83614134b4e90: Bootstrap starting.
I20260619 02:40:30.018311 15861 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d6cad4dbc39438196d83614134b4e90: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:30.018610 15861 log.cc:824] T 00000000000000000000000000000000 P 7d6cad4dbc39438196d83614134b4e90: Log is configured to *not* fsync() on all Append() calls
I20260619 02:40:30.019336 15861 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d6cad4dbc39438196d83614134b4e90: No bootstrap required, opened a new log
I20260619 02:40:30.020280 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 15794
I20260619 02:40:30.020364 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/master-0/wal/instance
I20260619 02:40:30.020560 6085 external_mini_cluster.cc:1428] Setting key 4de1a2c0c3f3a7f25ea0c9e49d7f320d
I20260619 02:40:30.021298 15861 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d6cad4dbc39438196d83614134b4e90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d6cad4dbc39438196d83614134b4e90" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 43953 } }
I20260619 02:40:30.021471 15861 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d6cad4dbc39438196d83614134b4e90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:30.021504 15861 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d6cad4dbc39438196d83614134b4e90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d6cad4dbc39438196d83614134b4e90, State: Initialized, Role: FOLLOWER
I20260619 02:40:30.021629 15861 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d6cad4dbc39438196d83614134b4e90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d6cad4dbc39438196d83614134b4e90" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 43953 } }
I20260619 02:40:30.021713 15861 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d6cad4dbc39438196d83614134b4e90 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:40:30.021745 15861 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d6cad4dbc39438196d83614134b4e90 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:40:30.021821 15861 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d6cad4dbc39438196d83614134b4e90 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:30.022445 15861 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d6cad4dbc39438196d83614134b4e90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d6cad4dbc39438196d83614134b4e90" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 43953 } }
I20260619 02:40:30.022552 15861 leader_election.cc:302] T 00000000000000000000000000000000 P 7d6cad4dbc39438196d83614134b4e90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d6cad4dbc39438196d83614134b4e90; no voters:
I20260619 02:40:30.022754 15861 leader_election.cc:288] T 00000000000000000000000000000000 P 7d6cad4dbc39438196d83614134b4e90 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:40:30.022845 15866 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d6cad4dbc39438196d83614134b4e90 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:40:30.023027 15866 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d6cad4dbc39438196d83614134b4e90 [term 1 LEADER]: Becoming Leader. State: Replica: 7d6cad4dbc39438196d83614134b4e90, State: Running, Role: LEADER
I20260619 02:40:30.023030 15861 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d6cad4dbc39438196d83614134b4e90 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:40:30.023155 15866 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d6cad4dbc39438196d83614134b4e90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d6cad4dbc39438196d83614134b4e90" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 43953 } }
I20260619 02:40:30.023308 15865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:30.021120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37460 (local address 127.5.241.126:43953)
0619 02:40:30.021354 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:30.021363 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:30.021394 (+ 31us) server_negotiation.cc:407] Connection header received
0619 02:40:30.021503 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:30.021510 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:30.021556 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:30.021700 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:30.022410 (+ 710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:30.022414 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:30.022414 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:40:30.022419 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:30.022466 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:40:30.022481 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:30.022554 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:30.022754 (+ 200us) server_negotiation.cc:299] Negotiation successful
0619 02:40:30.022836 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":103,"threads_started":1}
I20260619 02:40:30.023733 15867 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d6cad4dbc39438196d83614134b4e90 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d6cad4dbc39438196d83614134b4e90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d6cad4dbc39438196d83614134b4e90" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 43953 } } }
I20260619 02:40:30.023867 15867 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d6cad4dbc39438196d83614134b4e90 [sys.catalog]: This master's current role is: LEADER
I20260619 02:40:30.023468 15866 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d6cad4dbc39438196d83614134b4e90 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d6cad4dbc39438196d83614134b4e90. Latest consensus state: current_term: 1 leader_uuid: "7d6cad4dbc39438196d83614134b4e90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d6cad4dbc39438196d83614134b4e90" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 43953 } } }
I20260619 02:40:30.024083 15866 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d6cad4dbc39438196d83614134b4e90 [sys.catalog]: This master's current role is: LEADER
W20260619 02:40:30.024834 15882 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7d6cad4dbc39438196d83614134b4e90: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:40:30.024876 15882 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:40:30.024936 15875 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:40:30.025089 15875 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:40:30.026453 15875 catalog_manager.cc:1347] Generated new cluster ID: ff30a0cb195b43708c3d86aea9652470
I20260619 02:40:30.026503 15875 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:40:30.043295 15875 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:40:30.043490 15875 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:40:30.053699 15875 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d6cad4dbc39438196d83614134b4e90: Generated new TSK 0
I20260619 02:40:30.053926 15875 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:40:30.057364 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.65:0
--local_ip_for_outbound_sockets=127.5.241.65
--webserver_interface=127.5.241.65
--webserver_port=0
--tserver_master_addrs=127.5.241.126:43953
--builtin_ntp_servers=127.5.241.84:42241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 02:40:30.136737 15885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:30.136987 15885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:30.137022 15885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:30.138219 15885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:30.138281 15885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:30.138321 15885 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.65
I20260619 02:40:30.139667 15885 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:42241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.241.65
--webserver_port=0
--tserver_master_addrs=127.5.241.126:43953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15885
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.65
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:40:30.139928 15885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:30.140213 15885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:30.146253 15890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:30.146288 15891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:30.146288 15893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:30.146885 15885 server_base.cc:1034] running on GCE node
I20260619 02:40:30.147111 15885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:30.147341 15885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:30.148495 15885 hybrid_clock.cc:648] HybridClock initialized: now 1781836830148483 us; error 30 us; skew 500 ppm
I20260619 02:40:30.149748 15885 webserver.cc:460] Webserver started at http://127.5.241.65:36237/ using document root <none> and password file <none>
I20260619 02:40:30.149989 15885 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:30.150060 15885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:30.150192 15885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:40:30.151309 15885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-0/data/instance:
uuid: "c1c6b4e73f4a44d2a92c26e91e871ff8"
format_stamp: "Formatted at 2026-06-19 02:40:30 on dist-test-slave-zpbg"
server_key: "93fe243a493d249d79b4309d1582f7f7"
server_key_iv: "84270531494fa28f83dbe80b3a45e131"
server_key_version: "encryptionkey@0"
I20260619 02:40:30.151659 15885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-0/wal/instance:
uuid: "c1c6b4e73f4a44d2a92c26e91e871ff8"
format_stamp: "Formatted at 2026-06-19 02:40:30 on dist-test-slave-zpbg"
server_key: "93fe243a493d249d79b4309d1582f7f7"
server_key_iv: "84270531494fa28f83dbe80b3a45e131"
server_key_version: "encryptionkey@0"
I20260619 02:40:30.152952 15885 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:40:30.153698 15899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:30.153868 15885 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:40:30.153965 15885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-0/wal
uuid: "c1c6b4e73f4a44d2a92c26e91e871ff8"
format_stamp: "Formatted at 2026-06-19 02:40:30 on dist-test-slave-zpbg"
server_key: "93fe243a493d249d79b4309d1582f7f7"
server_key_iv: "84270531494fa28f83dbe80b3a45e131"
server_key_version: "encryptionkey@0"
I20260619 02:40:30.154052 15885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:30.157389 15885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:30.157639 15885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:30.157779 15885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:30.157984 15885 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:40:30.158274 15885 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:40:30.158334 15885 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:30.158372 15885 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:40:30.158398 15885 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:30.164258 15885 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.65:33973
I20260619 02:40:30.164311 16012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.65:33973 every 8 connection(s)
I20260619 02:40:30.164635 15885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
I20260619 02:40:30.167577 15865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:30.165323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:47875 (local address 127.5.241.126:43953)
0619 02:40:30.165366 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:30.165368 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:30.165575 (+ 207us) server_negotiation.cc:407] Connection header received
0619 02:40:30.166069 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:30.166070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:30.166078 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:30.166134 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:30.167332 (+ 1198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:30.167333 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:30.167334 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:40:30.167338 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:30.167355 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:40:30.167359 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:30.167412 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:30.167492 (+ 80us) server_negotiation.cc:299] Negotiation successful
0619 02:40:30.167509 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 02:40:30.167760 16015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:30.165301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:43953 (local address 127.5.241.65:47875)
0619 02:40:30.165463 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:30.165495 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:30.165967 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:30.166206 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:30.166210 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:30.166219 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:30.167182 (+ 963us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:40:30.167182 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:30.167208 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:40:30.167235 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:30.167409 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:30.167414 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:30.167417 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:40:30.167492 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 02:40:30.167550 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":71,"threads_started":1}
I20260619 02:40:30.168262 16013 heartbeater.cc:347] Connected to a master server at 127.5.241.126:43953
I20260619 02:40:30.168351 16013 heartbeater.cc:464] Registering TS with master...
I20260619 02:40:30.168462 16013 heartbeater.cc:511] Master 127.5.241.126:43953 requested a full tablet report, sending...
I20260619 02:40:30.168931 15825 ts_manager.cc:194] Registered new tserver with Master: c1c6b4e73f4a44d2a92c26e91e871ff8 (127.5.241.65:33973)
I20260619 02:40:30.172159 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 15885
I20260619 02:40:30.172267 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-0/wal/instance
I20260619 02:40:30.172533 6085 external_mini_cluster.cc:1428] Setting key b9d40e1063170eb7539e1ab73fa8dddd
I20260619 02:40:30.173444 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.66:0
--local_ip_for_outbound_sockets=127.5.241.66
--webserver_interface=127.5.241.66
--webserver_port=0
--tserver_master_addrs=127.5.241.126:43953
--builtin_ntp_servers=127.5.241.84:42241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 02:40:30.254892 16016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:30.255178 16016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:30.255220 16016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:30.256547 16016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:30.256608 16016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:30.256654 16016 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.66
I20260619 02:40:30.258122 16016 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:42241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.241.66
--webserver_port=0
--tserver_master_addrs=127.5.241.126:43953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16016
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.66
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:40:30.258404 16016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:30.258713 16016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:30.263769 16021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:30.263759 16024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:30.264035 16022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:30.264043 16016 server_base.cc:1034] running on GCE node
I20260619 02:40:30.264302 16016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:30.264513 16016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:30.265652 16016 hybrid_clock.cc:648] HybridClock initialized: now 1781836830265642 us; error 28 us; skew 500 ppm
I20260619 02:40:30.267074 16016 webserver.cc:460] Webserver started at http://127.5.241.66:45801/ using document root <none> and password file <none>
I20260619 02:40:30.267279 16016 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:30.267323 16016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:30.267443 16016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:40:30.268515 16016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-1/data/instance:
uuid: "b341927e919b461ea27b8575dd76a649"
format_stamp: "Formatted at 2026-06-19 02:40:30 on dist-test-slave-zpbg"
server_key: "daca5155105119e72647b08f52b1e96f"
server_key_iv: "84721e6053567e3ad1c335b1f4e8cbdc"
server_key_version: "encryptionkey@0"
I20260619 02:40:30.268821 16016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-1/wal/instance:
uuid: "b341927e919b461ea27b8575dd76a649"
format_stamp: "Formatted at 2026-06-19 02:40:30 on dist-test-slave-zpbg"
server_key: "daca5155105119e72647b08f52b1e96f"
server_key_iv: "84721e6053567e3ad1c335b1f4e8cbdc"
server_key_version: "encryptionkey@0"
I20260619 02:40:30.270047 16016 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:40:30.270810 16030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:30.271003 16016 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:40:30.271103 16016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-1/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-1/wal
uuid: "b341927e919b461ea27b8575dd76a649"
format_stamp: "Formatted at 2026-06-19 02:40:30 on dist-test-slave-zpbg"
server_key: "daca5155105119e72647b08f52b1e96f"
server_key_iv: "84721e6053567e3ad1c335b1f4e8cbdc"
server_key_version: "encryptionkey@0"
I20260619 02:40:30.271193 16016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:30.274508 16016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:30.274832 16016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:30.274991 16016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:30.275199 16016 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:40:30.275513 16016 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:40:30.275565 16016 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:30.275604 16016 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:40:30.275630 16016 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:30.281613 16016 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.66:39283
I20260619 02:40:30.281654 16143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.66:39283 every 8 connection(s)
I20260619 02:40:30.282025 16016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
I20260619 02:40:30.285022 15865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:30.282828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:49265 (local address 127.5.241.126:43953)
0619 02:40:30.282875 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:30.282878 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:30.283112 (+ 234us) server_negotiation.cc:407] Connection header received
0619 02:40:30.283551 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:30.283552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:30.283558 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:30.283637 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:30.284811 (+ 1174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:30.284812 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:30.284812 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:40:30.284816 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:30.284831 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:40:30.284836 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:30.284897 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:30.284948 (+ 51us) server_negotiation.cc:299] Negotiation successful
0619 02:40:30.284960 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 02:40:30.285229 16146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:30.282776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:43953 (local address 127.5.241.66:49265)
0619 02:40:30.282963 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:30.283003 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:30.283453 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:30.283687 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:30.283690 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:30.283697 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:30.284659 (+ 962us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:40:30.284659 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:30.284688 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:40:30.284716 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:30.284897 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:30.284899 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:30.284902 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:40:30.284949 (+ 47us) client_negotiation.cc:240] Negotiation successful
0619 02:40:30.284995 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":79,"threads_started":1}
I20260619 02:40:30.285712 16144 heartbeater.cc:347] Connected to a master server at 127.5.241.126:43953
I20260619 02:40:30.285795 16144 heartbeater.cc:464] Registering TS with master...
I20260619 02:40:30.285959 16144 heartbeater.cc:511] Master 127.5.241.126:43953 requested a full tablet report, sending...
I20260619 02:40:30.286350 15825 ts_manager.cc:194] Registered new tserver with Master: b341927e919b461ea27b8575dd76a649 (127.5.241.66:39283)
I20260619 02:40:30.287856 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 16016
I20260619 02:40:30.287937 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-1/wal/instance
I20260619 02:40:30.288102 6085 external_mini_cluster.cc:1428] Setting key f0e07b7f3a7b33cd0c6d9aa5789bc345
I20260619 02:40:30.288950 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.67:0
--local_ip_for_outbound_sockets=127.5.241.67
--webserver_interface=127.5.241.67
--webserver_port=0
--tserver_master_addrs=127.5.241.126:43953
--builtin_ntp_servers=127.5.241.84:42241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 02:40:30.368813 16147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:30.369015 16147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:30.369037 16147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:30.370128 16147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:30.370162 16147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:30.370186 16147 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.67
I20260619 02:40:30.371433 16147 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:42241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.241.67
--webserver_port=0
--tserver_master_addrs=127.5.241.126:43953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.67
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:40:30.371618 16147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:30.371855 16147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:30.376677 16153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:30.376657 16155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:30.376874 16147 server_base.cc:1034] running on GCE node
W20260619 02:40:30.376907 16152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:30.377125 16147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:30.377360 16147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:30.378499 16147 hybrid_clock.cc:648] HybridClock initialized: now 1781836830378479 us; error 36 us; skew 500 ppm
I20260619 02:40:30.379746 16147 webserver.cc:460] Webserver started at http://127.5.241.67:44085/ using document root <none> and password file <none>
I20260619 02:40:30.379994 16147 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:30.380040 16147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:30.380206 16147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:40:30.381256 16147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-2/data/instance:
uuid: "c2023667a0cd4ccabaa05bb1deeeb2e9"
format_stamp: "Formatted at 2026-06-19 02:40:30 on dist-test-slave-zpbg"
server_key: "4b27425b460c742441c028fc2bec75a8"
server_key_iv: "cf02d97a2e3551bf68fb992f66e30086"
server_key_version: "encryptionkey@0"
I20260619 02:40:30.381601 16147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-2/wal/instance:
uuid: "c2023667a0cd4ccabaa05bb1deeeb2e9"
format_stamp: "Formatted at 2026-06-19 02:40:30 on dist-test-slave-zpbg"
server_key: "4b27425b460c742441c028fc2bec75a8"
server_key_iv: "cf02d97a2e3551bf68fb992f66e30086"
server_key_version: "encryptionkey@0"
I20260619 02:40:30.382997 16147 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:40:30.383692 16161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:30.383879 16147 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:40:30.383944 16147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-2/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-2/wal
uuid: "c2023667a0cd4ccabaa05bb1deeeb2e9"
format_stamp: "Formatted at 2026-06-19 02:40:30 on dist-test-slave-zpbg"
server_key: "4b27425b460c742441c028fc2bec75a8"
server_key_iv: "cf02d97a2e3551bf68fb992f66e30086"
server_key_version: "encryptionkey@0"
I20260619 02:40:30.384034 16147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:30.387286 16147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:30.387532 16147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:30.387665 16147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:30.387858 16147 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:40:30.388146 16147 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:40:30.388177 16147 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:30.388224 16147 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:40:30.388252 16147 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:30.393736 16147 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.67:38851
I20260619 02:40:30.393792 16274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.67:38851 every 8 connection(s)
I20260619 02:40:30.394112 16147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
I20260619 02:40:30.397012 15865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:30.394860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:43119 (local address 127.5.241.126:43953)
0619 02:40:30.394928 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:30.394930 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:30.395186 (+ 256us) server_negotiation.cc:407] Connection header received
0619 02:40:30.395623 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:30.395623 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:30.395628 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:30.395679 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:30.396846 (+ 1167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:30.396847 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:30.396847 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:40:30.396850 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:30.396859 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:40:30.396862 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:30.396906 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:30.396950 (+ 44us) server_negotiation.cc:299] Negotiation successful
0619 02:40:30.396961 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 02:40:30.397198 16277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:30.394825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:43953 (local address 127.5.241.67:43119)
0619 02:40:30.395049 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:30.395092 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:30.395543 (+ 451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:30.395740 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:30.395743 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:30.395751 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:30.396722 (+ 971us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:40:30.396722 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:30.396749 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:40:30.396776 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:30.396906 (+ 130us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:30.396907 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:30.396909 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 02:40:30.396957 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 02:40:30.396999 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":102,"threads_started":1}
I20260619 02:40:30.397621 16275 heartbeater.cc:347] Connected to a master server at 127.5.241.126:43953
I20260619 02:40:30.397710 16275 heartbeater.cc:464] Registering TS with master...
I20260619 02:40:30.397817 16275 heartbeater.cc:511] Master 127.5.241.126:43953 requested a full tablet report, sending...
I20260619 02:40:30.398182 15825 ts_manager.cc:194] Registered new tserver with Master: c2023667a0cd4ccabaa05bb1deeeb2e9 (127.5.241.67:38851)
I20260619 02:40:30.403605 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 16147
I20260619 02:40:30.403688 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836772852275-6085-0/minicluster-data/ts-2/wal/instance
I20260619 02:40:30.403838 6085 external_mini_cluster.cc:1428] Setting key 610d68716c265e0e6bea02d601c65f82
I20260619 02:40:30.404708 6085 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 02:40:30.406435 15865 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:30.405992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37470 (local address 127.5.241.126:43953)
0619 02:40:30.406033 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:30.406035 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:30.406151 (+ 116us) server_negotiation.cc:407] Connection header received
0619 02:40:30.406191 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:30.406192 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:30.406196 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:30.406241 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:30.406369 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37470: BlockingRecv error: recv got EOF from 127.0.0.1:37470 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 02:40:30.406514 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 15885
I20260619 02:40:30.411083 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 16016
I20260619 02:40:30.415530 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 16147
I20260619 02:40:30.419612 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 15794
2026-06-19T02:40:30Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (540 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:40:30 dist-test-slave-zpbg krb5kdc[16285](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:40:30 dist-test-slave-zpbg krb5kdc[16285](info): set up 2 sockets
Jun 19 02:40:30 dist-test-slave-zpbg krb5kdc[16285](info): commencing operation
krb5kdc: starting...
W20260619 02:40:31.455669 6085 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Jun 19 02:40:31 dist-test-slave-zpbg krb5kdc[16285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:40:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:40:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.241.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
I20260619 02:40:31.574247 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:37859
--webserver_interface=127.5.241.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:39779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:37859
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:40:31.656426 16298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:31.656644 16298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:31.656667 16298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:31.657915 16298 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:40:31.657979 16298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:31.657996 16298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:31.658008 16298 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:40:31.658049 16298 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:40:31.660320 16298 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:39779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:37859
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:37859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.241.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16298
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:40:31.660600 16298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:31.660866 16298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:40:31.665939 16298 server_base.cc:1034] running on GCE node
W20260619 02:40:31.665920 16303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:31.666096 16306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:31.665915 16304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:31.666371 16298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:31.666718 16298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:31.667908 16298 hybrid_clock.cc:648] HybridClock initialized: now 1781836831667881 us; error 36 us; skew 500 ppm
Jun 19 02:40:31 dist-test-slave-zpbg krb5kdc[16285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836831, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:31.670450 16298 init.cc:377] Logged in from keytab as kudu/127.5.241.126@KRBTEST.COM (short username kudu)
I20260619 02:40:31.670935 16298 webserver.cc:460] Webserver started at http://127.5.241.126:37239/ using document root <none> and password file <none>
I20260619 02:40:31.671175 16298 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:31.671244 16298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:31.671378 16298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:40:31.672492 16298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/master-0/data/instance:
uuid: "ecdb0f02f4c44ecfb00238ec0b01eff2"
format_stamp: "Formatted at 2026-06-19 02:40:31 on dist-test-slave-zpbg"
server_key: "fd8aee7225d4384aae873e48d27d2c48"
server_key_iv: "78d4b98f593640ee16d2b6718c6d7dba"
server_key_version: "encryptionkey@0"
I20260619 02:40:31.672864 16298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/master-0/wal/instance:
uuid: "ecdb0f02f4c44ecfb00238ec0b01eff2"
format_stamp: "Formatted at 2026-06-19 02:40:31 on dist-test-slave-zpbg"
server_key: "fd8aee7225d4384aae873e48d27d2c48"
server_key_iv: "78d4b98f593640ee16d2b6718c6d7dba"
server_key_version: "encryptionkey@0"
I20260619 02:40:31.674163 16298 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:40:31.674957 16313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:31.675124 16298 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:40:31.675230 16298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "ecdb0f02f4c44ecfb00238ec0b01eff2"
format_stamp: "Formatted at 2026-06-19 02:40:31 on dist-test-slave-zpbg"
server_key: "fd8aee7225d4384aae873e48d27d2c48"
server_key_iv: "78d4b98f593640ee16d2b6718c6d7dba"
server_key_version: "encryptionkey@0"
I20260619 02:40:31.675302 16298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:31.692863 16298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:31.695602 16298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:31.695768 16298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:31.699604 16298 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.126:37859
I20260619 02:40:31.699635 16365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.126:37859 every 8 connection(s)
I20260619 02:40:31.699982 16298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:40:31.700901 16366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:31.703146 16366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecdb0f02f4c44ecfb00238ec0b01eff2: Bootstrap starting.
I20260619 02:40:31.703713 16366 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ecdb0f02f4c44ecfb00238ec0b01eff2: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:31.703997 16366 log.cc:824] T 00000000000000000000000000000000 P ecdb0f02f4c44ecfb00238ec0b01eff2: Log is configured to *not* fsync() on all Append() calls
I20260619 02:40:31.704684 16366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecdb0f02f4c44ecfb00238ec0b01eff2: No bootstrap required, opened a new log
I20260619 02:40:31.706218 16366 raft_consensus.cc:348] T 00000000000000000000000000000000 P ecdb0f02f4c44ecfb00238ec0b01eff2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecdb0f02f4c44ecfb00238ec0b01eff2" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 37859 } }
I20260619 02:40:31.706349 16366 raft_consensus.cc:374] T 00000000000000000000000000000000 P ecdb0f02f4c44ecfb00238ec0b01eff2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:31.706372 16366 raft_consensus.cc:729] T 00000000000000000000000000000000 P ecdb0f02f4c44ecfb00238ec0b01eff2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecdb0f02f4c44ecfb00238ec0b01eff2, State: Initialized, Role: FOLLOWER
I20260619 02:40:31.706535 16366 consensus_queue.cc:260] T 00000000000000000000000000000000 P ecdb0f02f4c44ecfb00238ec0b01eff2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecdb0f02f4c44ecfb00238ec0b01eff2" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 37859 } }
I20260619 02:40:31.706606 16366 raft_consensus.cc:388] T 00000000000000000000000000000000 P ecdb0f02f4c44ecfb00238ec0b01eff2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:40:31.706627 16366 raft_consensus.cc:482] T 00000000000000000000000000000000 P ecdb0f02f4c44ecfb00238ec0b01eff2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:40:31.706655 16366 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ecdb0f02f4c44ecfb00238ec0b01eff2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:31.707502 16366 raft_consensus.cc:504] T 00000000000000000000000000000000 P ecdb0f02f4c44ecfb00238ec0b01eff2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecdb0f02f4c44ecfb00238ec0b01eff2" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 37859 } }
I20260619 02:40:31.707587 16366 leader_election.cc:302] T 00000000000000000000000000000000 P ecdb0f02f4c44ecfb00238ec0b01eff2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ecdb0f02f4c44ecfb00238ec0b01eff2; no voters:
I20260619 02:40:31.707754 16366 leader_election.cc:288] T 00000000000000000000000000000000 P ecdb0f02f4c44ecfb00238ec0b01eff2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:40:31.707801 16369 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ecdb0f02f4c44ecfb00238ec0b01eff2 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:40:31.707973 16369 raft_consensus.cc:686] T 00000000000000000000000000000000 P ecdb0f02f4c44ecfb00238ec0b01eff2 [term 1 LEADER]: Becoming Leader. State: Replica: ecdb0f02f4c44ecfb00238ec0b01eff2, State: Running, Role: LEADER
I20260619 02:40:31.708055 16369 consensus_queue.cc:237] T 00000000000000000000000000000000 P ecdb0f02f4c44ecfb00238ec0b01eff2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecdb0f02f4c44ecfb00238ec0b01eff2" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 37859 } }
I20260619 02:40:31.708061 16366 sys_catalog.cc:564] T 00000000000000000000000000000000 P ecdb0f02f4c44ecfb00238ec0b01eff2 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:40:31.708503 16371 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecdb0f02f4c44ecfb00238ec0b01eff2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ecdb0f02f4c44ecfb00238ec0b01eff2. Latest consensus state: current_term: 1 leader_uuid: "ecdb0f02f4c44ecfb00238ec0b01eff2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecdb0f02f4c44ecfb00238ec0b01eff2" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 37859 } } }
I20260619 02:40:31.708487 16370 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecdb0f02f4c44ecfb00238ec0b01eff2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ecdb0f02f4c44ecfb00238ec0b01eff2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecdb0f02f4c44ecfb00238ec0b01eff2" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 37859 } } }
I20260619 02:40:31.708642 16371 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecdb0f02f4c44ecfb00238ec0b01eff2 [sys.catalog]: This master's current role is: LEADER
I20260619 02:40:31.708698 16370 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecdb0f02f4c44ecfb00238ec0b01eff2 [sys.catalog]: This master's current role is: LEADER
I20260619 02:40:31.708985 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 16298
I20260619 02:40:31.709103 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/master-0/wal/instance
I20260619 02:40:31.709336 6085 external_mini_cluster.cc:1428] Setting key d7a0c4580ffe126084ad1462f8570662
W20260619 02:40:31.710090 16384 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ecdb0f02f4c44ecfb00238ec0b01eff2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:40:31.710181 16384 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:40:31.710285 16375 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:40:31.710490 16375 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:40:31.712150 16375 catalog_manager.cc:1347] Generated new cluster ID: 844809cf6fc44e8588abbe4aa168f697
I20260619 02:40:31.712204 16375 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 02:40:31 dist-test-slave-zpbg krb5kdc[16285](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:40:31.720764 16388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:31.709818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37130 (local address 127.5.241.126:37859)
0619 02:40:31.709982 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:31.709990 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:31.710155 (+ 165us) server_negotiation.cc:407] Connection header received
0619 02:40:31.710317 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:31.710325 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:31.710385 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:31.710539 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:31.711288 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:31.712493 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:31.713567 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:31.714156 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:31.714255 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:31.717045 (+ 2790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:31.717051 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:31.717052 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:31.717068 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:31.719117 (+ 2049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:31.719508 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:31.719512 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:31.719512 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:31.719638 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:31.719838 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:31.719839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:31.719840 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:31.720173 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:31.720331 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:31.720408 (+ 77us) server_negotiation.cc:299] Negotiation successful
0619 02:40:31.720552 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":73,"threads_started":1}
I20260619 02:40:31.730150 16375 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:40:31.730729 16375 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:40:31.742657 16375 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ecdb0f02f4c44ecfb00238ec0b01eff2: Generated new TSK 0
I20260619 02:40:31.742895 16375 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.241.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab.
Entry for principal kudu/127.5.241.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab.
I20260619 02:40:31.773314 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.65:0
--local_ip_for_outbound_sockets=127.5.241.65
--webserver_interface=127.5.241.65
--webserver_port=0
--tserver_master_addrs=127.5.241.126:37859
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab
--principal=kudu/127.5.241.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:39779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:40:31.857744 16392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:31.857960 16392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:31.857986 16392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:31.859162 16392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:31.859200 16392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:31.859227 16392 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.65
I20260619 02:40:31.860631 16392 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:39779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab
--principal=kudu/127.5.241.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.241.65
--webserver_port=0
--tserver_master_addrs=127.5.241.126:37859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16392
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.65
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:40:31.860833 16392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:31.861035 16392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:31.866303 16400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:31.866320 16397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:31.866389 16398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:31.867043 16392 server_base.cc:1034] running on GCE node
I20260619 02:40:31.867306 16392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:31.867570 16392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:31.868736 16392 hybrid_clock.cc:648] HybridClock initialized: now 1781836831868728 us; error 37 us; skew 500 ppm
Jun 19 02:40:31 dist-test-slave-zpbg krb5kdc[16285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836831, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:31.871620 16392 init.cc:377] Logged in from keytab as kudu/127.5.241.65@KRBTEST.COM (short username kudu)
I20260619 02:40:31.872098 16392 webserver.cc:460] Webserver started at http://127.5.241.65:42637/ using document root <none> and password file <none>
I20260619 02:40:31.872344 16392 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:31.872416 16392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:31.872658 16392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:40:31.873782 16392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/ts-0/data/instance:
uuid: "301da70e9f6c4b929b03b669a15187f1"
format_stamp: "Formatted at 2026-06-19 02:40:31 on dist-test-slave-zpbg"
server_key: "5dc26a9db9c97c27da113c66ff6163c8"
server_key_iv: "2407490ca6ac9a76b4db6e42fc2e8de1"
server_key_version: "encryptionkey@0"
I20260619 02:40:31.874171 16392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/ts-0/wal/instance:
uuid: "301da70e9f6c4b929b03b669a15187f1"
format_stamp: "Formatted at 2026-06-19 02:40:31 on dist-test-slave-zpbg"
server_key: "5dc26a9db9c97c27da113c66ff6163c8"
server_key_iv: "2407490ca6ac9a76b4db6e42fc2e8de1"
server_key_version: "encryptionkey@0"
I20260619 02:40:31.875720 16392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:40:31.876600 16407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:31.876807 16392 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:40:31.876929 16392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/ts-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/ts-0/wal
uuid: "301da70e9f6c4b929b03b669a15187f1"
format_stamp: "Formatted at 2026-06-19 02:40:31 on dist-test-slave-zpbg"
server_key: "5dc26a9db9c97c27da113c66ff6163c8"
server_key_iv: "2407490ca6ac9a76b4db6e42fc2e8de1"
server_key_version: "encryptionkey@0"
I20260619 02:40:31.877007 16392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:31.903757 16392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:31.905834 16392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:31.906013 16392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:31.906270 16392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:40:31.906589 16392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:40:31.906644 16392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:31.906687 16392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:40:31.906730 16392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:31.913364 16392 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.65:45817
I20260619 02:40:31.913414 16520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.65:45817 every 8 connection(s)
I20260619 02:40:31.913769 16392 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
I20260619 02:40:31.918623 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 16392
I20260619 02:40:31.918763 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/ts-0/wal/instance
I20260619 02:40:31.918982 6085 external_mini_cluster.cc:1428] Setting key 77e840b793e3560df03b164cd54b49e2
Jun 19 02:40:31 dist-test-slave-zpbg krb5kdc[16285](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836831, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.65@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:40:31.925657 16388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:31.914763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:43049 (local address 127.5.241.126:37859)
0619 02:40:31.914849 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:31.914851 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:31.915090 (+ 239us) server_negotiation.cc:407] Connection header received
0619 02:40:31.915512 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:31.915513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:31.915528 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:31.915595 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:31.916613 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:31.917142 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:31.918324 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:31.918543 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:31.918659 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:31.921207 (+ 2548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:31.921209 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:31.921210 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:31.921227 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:31.923247 (+ 2020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:31.923708 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:31.923710 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:31.923710 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:31.923794 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:31.924643 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:31.924644 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:31.924645 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:31.924816 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:31.924916 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:31.925375 (+ 459us) server_negotiation.cc:299] Negotiation successful
0619 02:40:31.925513 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 02:40:31.925788 16523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:31.914682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:37859 (local address 127.5.241.65:43049)
0619 02:40:31.914846 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:31.914901 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:31.915403 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:31.915670 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:31.915675 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:31.916300 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:31.916506 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:31.916511 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:31.917376 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:31.917377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:31.918206 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:31.918208 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:31.918667 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:31.918668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:31.918944 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:31.919639 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:31.919650 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:31.921079 (+ 1429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:31.923343 (+ 2264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:31.923350 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:31.923352 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:31.923618 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:31.923931 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:31.923933 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:31.923933 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:31.924543 (+ 610us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:31.924937 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:31.924943 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:31.925208 (+ 265us) client_negotiation.cc:769] Sending connection context
0619 02:40:31.925321 (+ 113us) client_negotiation.cc:240] Negotiation successful
0619 02:40:31.925542 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":107,"threads_started":1}
I20260619 02:40:31.926448 16521 heartbeater.cc:347] Connected to a master server at 127.5.241.126:37859
I20260619 02:40:31.926587 16521 heartbeater.cc:464] Registering TS with master...
I20260619 02:40:31.926863 16521 heartbeater.cc:511] Master 127.5.241.126:37859 requested a full tablet report, sending...
I20260619 02:40:31.927589 16330 ts_manager.cc:194] Registered new tserver with Master: 301da70e9f6c4b929b03b669a15187f1 (127.5.241.65:45817)
I20260619 02:40:31.928553 16330 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.65@KRBTEST.COM'} at 127.5.241.65:43049
WARNING: no policy specified for kudu/127.5.241.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab.
Entry for principal kudu/127.5.241.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab.
I20260619 02:40:31.946195 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.66:0
--local_ip_for_outbound_sockets=127.5.241.66
--webserver_interface=127.5.241.66
--webserver_port=0
--tserver_master_addrs=127.5.241.126:37859
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab
--principal=kudu/127.5.241.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:39779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:40:32.026335 16526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:32.026551 16526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:32.026574 16526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:32.027784 16526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:32.027827 16526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:32.027853 16526 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.66
I20260619 02:40:32.029309 16526 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:39779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab
--principal=kudu/127.5.241.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.241.66
--webserver_port=0
--tserver_master_addrs=127.5.241.126:37859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16526
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.66
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:40:32.029551 16526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:32.029798 16526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:32.034848 16531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:32.034837 16532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:32.034950 16526 server_base.cc:1034] running on GCE node
W20260619 02:40:32.035058 16534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:32.035291 16526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:32.035574 16526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:32.036711 16526 hybrid_clock.cc:648] HybridClock initialized: now 1781836832036694 us; error 45 us; skew 500 ppm
Jun 19 02:40:32 dist-test-slave-zpbg krb5kdc[16285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836832, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:32.039548 16526 init.cc:377] Logged in from keytab as kudu/127.5.241.66@KRBTEST.COM (short username kudu)
I20260619 02:40:32.040000 16526 webserver.cc:460] Webserver started at http://127.5.241.66:41143/ using document root <none> and password file <none>
I20260619 02:40:32.040232 16526 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:32.040279 16526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:32.040462 16526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:40:32.041558 16526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/ts-1/data/instance:
uuid: "eb298bf474a842dc83c5bed13d2ae53f"
format_stamp: "Formatted at 2026-06-19 02:40:32 on dist-test-slave-zpbg"
server_key: "68cc2abc05cf7f3daf3463ca6bbbfd60"
server_key_iv: "aca1804d5c0f87e5013dc7254dfbcfee"
server_key_version: "encryptionkey@0"
I20260619 02:40:32.041926 16526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/ts-1/wal/instance:
uuid: "eb298bf474a842dc83c5bed13d2ae53f"
format_stamp: "Formatted at 2026-06-19 02:40:32 on dist-test-slave-zpbg"
server_key: "68cc2abc05cf7f3daf3463ca6bbbfd60"
server_key_iv: "aca1804d5c0f87e5013dc7254dfbcfee"
server_key_version: "encryptionkey@0"
I20260619 02:40:32.043335 16526 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:40:32.044057 16541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:32.044257 16526 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:40:32.044330 16526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/ts-1/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/ts-1/wal
uuid: "eb298bf474a842dc83c5bed13d2ae53f"
format_stamp: "Formatted at 2026-06-19 02:40:32 on dist-test-slave-zpbg"
server_key: "68cc2abc05cf7f3daf3463ca6bbbfd60"
server_key_iv: "aca1804d5c0f87e5013dc7254dfbcfee"
server_key_version: "encryptionkey@0"
I20260619 02:40:32.044420 16526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:32.055850 16526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:32.058218 16526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:32.058373 16526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:32.058626 16526 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:40:32.058990 16526 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:40:32.059024 16526 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:32.059080 16526 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:40:32.059108 16526 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:32.064848 16526 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.66:45635
I20260619 02:40:32.064904 16654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.66:45635 every 8 connection(s)
I20260619 02:40:32.065219 16526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
I20260619 02:40:32.071002 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 16526
Jun 19 02:40:32 dist-test-slave-zpbg krb5kdc[16285](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836832, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.66@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:40:32.071127 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/ts-1/wal/instance
I20260619 02:40:32.071323 6085 external_mini_cluster.cc:1428] Setting key 42e600962fe55517851e49e04191d74a
I20260619 02:40:32.074666 16388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:32.065909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:48307 (local address 127.5.241.126:37859)
0619 02:40:32.065959 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:32.065962 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:32.066209 (+ 247us) server_negotiation.cc:407] Connection header received
0619 02:40:32.066678 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:32.066679 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:32.066690 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:32.066743 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:32.067741 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:32.068261 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:32.069246 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:32.069419 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:32.069477 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:32.071645 (+ 2168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:32.071647 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:32.071648 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:32.071655 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:32.073256 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:32.073689 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:32.073691 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:32.073691 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:32.073754 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:32.074010 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:32.074011 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:32.074011 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:32.074130 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:32.074204 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:32.074492 (+ 288us) server_negotiation.cc:299] Negotiation successful
0619 02:40:32.074574 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 02:40:32.074899 16657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:32.065909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:37859 (local address 127.5.241.66:48307)
0619 02:40:32.066093 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:32.066128 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:32.066598 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:32.066874 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:32.066879 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:32.067454 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:32.067644 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:32.067649 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:32.068380 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:32.068381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:32.069174 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:32.069176 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:32.069495 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:32.069496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:32.069596 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:32.070065 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:32.070074 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:32.071551 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:32.073359 (+ 1808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:32.073366 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:32.073369 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:32.073598 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:32.073821 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:32.073822 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:32.073822 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:32.073941 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:32.074197 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:32.074203 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:32.074346 (+ 143us) client_negotiation.cc:769] Sending connection context
0619 02:40:32.074470 (+ 124us) client_negotiation.cc:240] Negotiation successful
0619 02:40:32.074634 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":99,"threads_started":1}
I20260619 02:40:32.075369 16655 heartbeater.cc:347] Connected to a master server at 127.5.241.126:37859
I20260619 02:40:32.075462 16655 heartbeater.cc:464] Registering TS with master...
I20260619 02:40:32.075656 16655 heartbeater.cc:511] Master 127.5.241.126:37859 requested a full tablet report, sending...
I20260619 02:40:32.076002 16330 ts_manager.cc:194] Registered new tserver with Master: eb298bf474a842dc83c5bed13d2ae53f (127.5.241.66:45635)
I20260619 02:40:32.076411 16330 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.66@KRBTEST.COM'} at 127.5.241.66:48307
WARNING: no policy specified for kudu/127.5.241.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab.
Entry for principal kudu/127.5.241.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab.
I20260619 02:40:32.099143 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.67:0
--local_ip_for_outbound_sockets=127.5.241.67
--webserver_interface=127.5.241.67
--webserver_port=0
--tserver_master_addrs=127.5.241.126:37859
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab
--principal=kudu/127.5.241.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:39779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:40:32.182041 16660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:32.182300 16660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:32.182327 16660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:32.183476 16660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:32.183517 16660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:32.183544 16660 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.67
I20260619 02:40:32.184900 16660 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:39779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab
--principal=kudu/127.5.241.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.241.67
--webserver_port=0
--tserver_master_addrs=127.5.241.126:37859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16660
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.67
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:40:32.185086 16660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:32.185263 16660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:32.190513 16665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:32.190538 16666 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:32.190547 16668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:32.191015 16660 server_base.cc:1034] running on GCE node
I20260619 02:40:32.191264 16660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:32.191521 16660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:32.192699 16660 hybrid_clock.cc:648] HybridClock initialized: now 1781836832192695 us; error 37 us; skew 500 ppm
Jun 19 02:40:32 dist-test-slave-zpbg krb5kdc[16285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836832, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:32.202595 16660 init.cc:377] Logged in from keytab as kudu/127.5.241.67@KRBTEST.COM (short username kudu)
I20260619 02:40:32.203262 16660 webserver.cc:460] Webserver started at http://127.5.241.67:43239/ using document root <none> and password file <none>
I20260619 02:40:32.203560 16660 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:32.203617 16660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:32.203742 16660 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:40:32.205183 16660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/ts-2/data/instance:
uuid: "8a75da29ae8046b7997d7e7ece52e36b"
format_stamp: "Formatted at 2026-06-19 02:40:32 on dist-test-slave-zpbg"
server_key: "25b9fea4edcc16119c255338c0af64b2"
server_key_iv: "bfaed35ac3608e9f528eea2cce8062e7"
server_key_version: "encryptionkey@0"
I20260619 02:40:32.205560 16660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/ts-2/wal/instance:
uuid: "8a75da29ae8046b7997d7e7ece52e36b"
format_stamp: "Formatted at 2026-06-19 02:40:32 on dist-test-slave-zpbg"
server_key: "25b9fea4edcc16119c255338c0af64b2"
server_key_iv: "bfaed35ac3608e9f528eea2cce8062e7"
server_key_version: "encryptionkey@0"
I20260619 02:40:32.207671 16660 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260619 02:40:32.209450 16675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:32.209614 16660 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 02:40:32.209702 16660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/ts-2/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/ts-2/wal
uuid: "8a75da29ae8046b7997d7e7ece52e36b"
format_stamp: "Formatted at 2026-06-19 02:40:32 on dist-test-slave-zpbg"
server_key: "25b9fea4edcc16119c255338c0af64b2"
server_key_iv: "bfaed35ac3608e9f528eea2cce8062e7"
server_key_version: "encryptionkey@0"
I20260619 02:40:32.209810 16660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:32.236284 16660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:32.238556 16660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:32.238679 16660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:32.239010 16660 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:40:32.239369 16660 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:40:32.239434 16660 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:32.239485 16660 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:40:32.239514 16660 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:32.247292 16660 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.67:46321
I20260619 02:40:32.247342 16788 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.67:46321 every 8 connection(s)
I20260619 02:40:32.247661 16660 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
Jun 19 02:40:32 dist-test-slave-zpbg krb5kdc[16285](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836832, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.67@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:40:32.255332 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 16660
I20260619 02:40:32.255415 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/ts-2/wal/instance
I20260619 02:40:32.255594 6085 external_mini_cluster.cc:1428] Setting key 0f93d48ec7e63c3bb60f7912ea854e98
I20260619 02:40:32.257061 16388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:32.248356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:52917 (local address 127.5.241.126:37859)
0619 02:40:32.248433 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:32.248443 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:32.248669 (+ 226us) server_negotiation.cc:407] Connection header received
0619 02:40:32.249162 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:32.249163 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:32.249174 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:32.249228 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:32.250173 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:32.250665 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:32.251709 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:32.251888 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:32.251943 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:32.254152 (+ 2209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:32.254154 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:32.254154 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:32.254159 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:32.255638 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:32.256049 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:32.256051 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:32.256051 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:32.256113 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:32.256402 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:32.256403 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:32.256403 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:32.256527 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:32.256603 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:32.256876 (+ 273us) server_negotiation.cc:299] Negotiation successful
0619 02:40:32.256952 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 02:40:32.257261 16791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:32.248348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:37859 (local address 127.5.241.67:52917)
0619 02:40:32.248542 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:32.248578 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:32.249053 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:32.249306 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:32.249309 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:32.249900 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:32.250090 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:32.250095 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:32.250753 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:32.250754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:32.251637 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:32.251639 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:32.251963 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:32.251964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:32.252067 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:32.252658 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:32.252667 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:32.254062 (+ 1395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:32.255716 (+ 1654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:32.255722 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:32.255724 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:32.255955 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:32.256208 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:32.256208 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:32.256209 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:32.256346 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:32.256593 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:32.256598 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:32.256741 (+ 143us) client_negotiation.cc:769] Sending connection context
0619 02:40:32.256854 (+ 113us) client_negotiation.cc:240] Negotiation successful
0619 02:40:32.257032 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":100,"threads_started":1}
I20260619 02:40:32.257748 16789 heartbeater.cc:347] Connected to a master server at 127.5.241.126:37859
I20260619 02:40:32.257826 16789 heartbeater.cc:464] Registering TS with master...
I20260619 02:40:32.258025 16789 heartbeater.cc:511] Master 127.5.241.126:37859 requested a full tablet report, sending...
I20260619 02:40:32.258471 16330 ts_manager.cc:194] Registered new tserver with Master: 8a75da29ae8046b7997d7e7ece52e36b (127.5.241.67:46321)
I20260619 02:40:32.258939 16330 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.67@KRBTEST.COM'} at 127.5.241.67:52917
I20260619 02:40:32.267254 6085 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:40:32.273856 16388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:32.268483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37132 (local address 127.5.241.126:37859)
0619 02:40:32.268554 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:32.268556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:32.268601 (+ 45us) server_negotiation.cc:407] Connection header received
0619 02:40:32.268646 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:32.268647 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:32.268655 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:32.268702 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:32.269220 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:32.269680 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:32.270361 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:32.270533 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:32.270581 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:32.271718 (+ 1137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:32.271719 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:32.271720 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:32.271726 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:32.272903 (+ 1177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:32.273203 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:32.273204 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:32.273204 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:32.273249 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:32.273401 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:32.273401 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:32.273402 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:32.273533 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:32.273605 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:32.273699 (+ 94us) server_negotiation.cc:299] Negotiation successful
0619 02:40:32.273754 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 02:40:32.275527 16330 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37132:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:40:32.277231 16330 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:40:32.283941 16802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:32.279649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:45817 (local address 127.0.0.1:55022)
0619 02:40:32.279803 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:32.279813 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:32.279884 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:32.280455 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:32.280456 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:32.280458 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:40:32.280491 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:32.280493 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:32.281476 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:32.281477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:32.282698 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:32.282700 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:32.283646 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:32.283647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:32.283730 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:32.283746 (+ 16us) client_negotiation.cc:769] Sending connection context
0619 02:40:32.283814 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 02:40:32.283865 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":4,"thread_start_us":89,"threads_started":1}
I20260619 02:40:32.284039 16803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:32.279878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42008 (local address 127.5.241.67:46321)
0619 02:40:32.280043 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:32.280048 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:32.280099 (+ 51us) server_negotiation.cc:407] Connection header received
0619 02:40:32.280174 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:32.280176 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:32.280194 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:32.280274 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:40:32.280565 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:32.281711 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:32.283042 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:32.283713 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:32.283774 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:32.283820 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:32.283923 (+ 103us) server_negotiation.cc:299] Negotiation successful
0619 02:40:32.283957 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":89,"threads_started":1}
I20260619 02:40:32.284041 16801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:32.279671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.67:46321 (local address 127.0.0.1:42008)
0619 02:40:32.280042 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:32.280053 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:32.280109 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:32.280284 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:32.280288 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:32.280291 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:40:32.280354 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:32.280356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:32.281805 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:32.281806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:32.282960 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:32.282961 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:32.283777 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:32.283778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:32.283843 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:32.283848 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 02:40:32.283910 (+ 62us) client_negotiation.cc:240] Negotiation successful
0619 02:40:32.283952 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":74,"threads_started":1}
I20260619 02:40:32.284262 16804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:32.279902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55022 (local address 127.5.241.65:45817)
0619 02:40:32.280080 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:32.280084 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:32.280095 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:40:32.280136 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:32.280138 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:32.280155 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:32.280261 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:40:32.280560 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:32.281382 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:32.282822 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:32.283555 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:32.283631 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:32.283688 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:32.284143 (+ 455us) server_negotiation.cc:299] Negotiation successful
0619 02:40:32.284188 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":96,"threads_started":1}
I20260619 02:40:32.284848 16800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:32.279646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.66:45635 (local address 127.0.0.1:36250)
0619 02:40:32.279744 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:32.279782 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:32.279846 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:32.280504 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:32.280505 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:32.280507 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:40:32.280541 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:32.280543 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:32.281798 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:32.281799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:32.283674 (+ 1875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:32.283676 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:32.284624 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:32.284625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:32.284688 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:32.284692 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:40:32.284750 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 02:40:32.284785 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75,"mutex_wait_us":1126,"thread_start_us":59,"threads_started":1}
I20260619 02:40:32.284883 16805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:32.280100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36250 (local address 127.5.241.66:45635)
0619 02:40:32.280255 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:32.280259 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:32.280269 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:40:32.280305 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:32.280307 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:32.280322 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:32.280403 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:40:32.280620 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:32.281675 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:32.283764 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:32.284494 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:32.284620 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:32.284676 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:32.284762 (+ 86us) server_negotiation.cc:299] Negotiation successful
0619 02:40:32.284798 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":84,"threads_started":1}
I20260619 02:40:32.284976 16455 tablet_service.cc:1467] Processing CreateTablet for tablet 62872effdc9a444cafc2fba4fa1bb46f (DEFAULT_TABLE table=test-table [id=4db4dfb588b242d29f3653d01024e9b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:40:32.285091 16723 tablet_service.cc:1467] Processing CreateTablet for tablet 62872effdc9a444cafc2fba4fa1bb46f (DEFAULT_TABLE table=test-table [id=4db4dfb588b242d29f3653d01024e9b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:40:32.285327 16455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62872effdc9a444cafc2fba4fa1bb46f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:32.285336 16723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62872effdc9a444cafc2fba4fa1bb46f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:32.285434 16589 tablet_service.cc:1467] Processing CreateTablet for tablet 62872effdc9a444cafc2fba4fa1bb46f (DEFAULT_TABLE table=test-table [id=4db4dfb588b242d29f3653d01024e9b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:40:32.285668 16589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62872effdc9a444cafc2fba4fa1bb46f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:32.288383 16806 tablet_bootstrap.cc:492] T 62872effdc9a444cafc2fba4fa1bb46f P eb298bf474a842dc83c5bed13d2ae53f: Bootstrap starting.
I20260619 02:40:32.288717 16808 tablet_bootstrap.cc:492] T 62872effdc9a444cafc2fba4fa1bb46f P 8a75da29ae8046b7997d7e7ece52e36b: Bootstrap starting.
I20260619 02:40:32.288851 16807 tablet_bootstrap.cc:492] T 62872effdc9a444cafc2fba4fa1bb46f P 301da70e9f6c4b929b03b669a15187f1: Bootstrap starting.
I20260619 02:40:32.289117 16806 tablet_bootstrap.cc:654] T 62872effdc9a444cafc2fba4fa1bb46f P eb298bf474a842dc83c5bed13d2ae53f: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:32.289376 16808 tablet_bootstrap.cc:654] T 62872effdc9a444cafc2fba4fa1bb46f P 8a75da29ae8046b7997d7e7ece52e36b: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:32.289423 16807 tablet_bootstrap.cc:654] T 62872effdc9a444cafc2fba4fa1bb46f P 301da70e9f6c4b929b03b669a15187f1: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:32.289722 16806 log.cc:824] T 62872effdc9a444cafc2fba4fa1bb46f P eb298bf474a842dc83c5bed13d2ae53f: Log is configured to *not* fsync() on all Append() calls
I20260619 02:40:32.289997 16808 log.cc:824] T 62872effdc9a444cafc2fba4fa1bb46f P 8a75da29ae8046b7997d7e7ece52e36b: Log is configured to *not* fsync() on all Append() calls
I20260619 02:40:32.289997 16807 log.cc:824] T 62872effdc9a444cafc2fba4fa1bb46f P 301da70e9f6c4b929b03b669a15187f1: Log is configured to *not* fsync() on all Append() calls
I20260619 02:40:32.290464 16806 tablet_bootstrap.cc:492] T 62872effdc9a444cafc2fba4fa1bb46f P eb298bf474a842dc83c5bed13d2ae53f: No bootstrap required, opened a new log
I20260619 02:40:32.290535 16806 ts_tablet_manager.cc:1397] T 62872effdc9a444cafc2fba4fa1bb46f P eb298bf474a842dc83c5bed13d2ae53f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:40:32.290695 16807 tablet_bootstrap.cc:492] T 62872effdc9a444cafc2fba4fa1bb46f P 301da70e9f6c4b929b03b669a15187f1: No bootstrap required, opened a new log
I20260619 02:40:32.290706 16808 tablet_bootstrap.cc:492] T 62872effdc9a444cafc2fba4fa1bb46f P 8a75da29ae8046b7997d7e7ece52e36b: No bootstrap required, opened a new log
I20260619 02:40:32.290788 16808 ts_tablet_manager.cc:1397] T 62872effdc9a444cafc2fba4fa1bb46f P 8a75da29ae8046b7997d7e7ece52e36b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:40:32.290773 16807 ts_tablet_manager.cc:1397] T 62872effdc9a444cafc2fba4fa1bb46f P 301da70e9f6c4b929b03b669a15187f1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:40:32.292069 16807 raft_consensus.cc:348] T 62872effdc9a444cafc2fba4fa1bb46f P 301da70e9f6c4b929b03b669a15187f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301da70e9f6c4b929b03b669a15187f1" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 45817 } } peers { permanent_uuid: "eb298bf474a842dc83c5bed13d2ae53f" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 45635 } } peers { permanent_uuid: "8a75da29ae8046b7997d7e7ece52e36b" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 46321 } }
I20260619 02:40:32.292271 16807 raft_consensus.cc:374] T 62872effdc9a444cafc2fba4fa1bb46f P 301da70e9f6c4b929b03b669a15187f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:32.292316 16807 raft_consensus.cc:729] T 62872effdc9a444cafc2fba4fa1bb46f P 301da70e9f6c4b929b03b669a15187f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 301da70e9f6c4b929b03b669a15187f1, State: Initialized, Role: FOLLOWER
I20260619 02:40:32.292423 16807 consensus_queue.cc:260] T 62872effdc9a444cafc2fba4fa1bb46f P 301da70e9f6c4b929b03b669a15187f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301da70e9f6c4b929b03b669a15187f1" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 45817 } } peers { permanent_uuid: "eb298bf474a842dc83c5bed13d2ae53f" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 45635 } } peers { permanent_uuid: "8a75da29ae8046b7997d7e7ece52e36b" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 46321 } }
I20260619 02:40:32.292491 16806 raft_consensus.cc:348] T 62872effdc9a444cafc2fba4fa1bb46f P eb298bf474a842dc83c5bed13d2ae53f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301da70e9f6c4b929b03b669a15187f1" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 45817 } } peers { permanent_uuid: "eb298bf474a842dc83c5bed13d2ae53f" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 45635 } } peers { permanent_uuid: "8a75da29ae8046b7997d7e7ece52e36b" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 46321 } }
I20260619 02:40:32.292652 16806 raft_consensus.cc:374] T 62872effdc9a444cafc2fba4fa1bb46f P eb298bf474a842dc83c5bed13d2ae53f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:32.292702 16806 raft_consensus.cc:729] T 62872effdc9a444cafc2fba4fa1bb46f P eb298bf474a842dc83c5bed13d2ae53f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb298bf474a842dc83c5bed13d2ae53f, State: Initialized, Role: FOLLOWER
I20260619 02:40:32.292716 16807 ts_tablet_manager.cc:1428] T 62872effdc9a444cafc2fba4fa1bb46f P 301da70e9f6c4b929b03b669a15187f1: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 02:40:32.292634 16808 raft_consensus.cc:348] T 62872effdc9a444cafc2fba4fa1bb46f P 8a75da29ae8046b7997d7e7ece52e36b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301da70e9f6c4b929b03b669a15187f1" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 45817 } } peers { permanent_uuid: "eb298bf474a842dc83c5bed13d2ae53f" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 45635 } } peers { permanent_uuid: "8a75da29ae8046b7997d7e7ece52e36b" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 46321 } }
I20260619 02:40:32.292783 16808 raft_consensus.cc:374] T 62872effdc9a444cafc2fba4fa1bb46f P 8a75da29ae8046b7997d7e7ece52e36b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:32.292841 16808 raft_consensus.cc:729] T 62872effdc9a444cafc2fba4fa1bb46f P 8a75da29ae8046b7997d7e7ece52e36b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a75da29ae8046b7997d7e7ece52e36b, State: Initialized, Role: FOLLOWER
I20260619 02:40:32.292847 16806 consensus_queue.cc:260] T 62872effdc9a444cafc2fba4fa1bb46f P eb298bf474a842dc83c5bed13d2ae53f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301da70e9f6c4b929b03b669a15187f1" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 45817 } } peers { permanent_uuid: "eb298bf474a842dc83c5bed13d2ae53f" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 45635 } } peers { permanent_uuid: "8a75da29ae8046b7997d7e7ece52e36b" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 46321 } }
I20260619 02:40:32.292882 16521 heartbeater.cc:503] Master 127.5.241.126:37859 was elected leader, sending a full tablet report...
I20260619 02:40:32.292951 16808 consensus_queue.cc:260] T 62872effdc9a444cafc2fba4fa1bb46f P 8a75da29ae8046b7997d7e7ece52e36b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301da70e9f6c4b929b03b669a15187f1" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 45817 } } peers { permanent_uuid: "eb298bf474a842dc83c5bed13d2ae53f" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 45635 } } peers { permanent_uuid: "8a75da29ae8046b7997d7e7ece52e36b" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 46321 } }
I20260619 02:40:32.293185 16655 heartbeater.cc:503] Master 127.5.241.126:37859 was elected leader, sending a full tablet report...
I20260619 02:40:32.293227 16808 ts_tablet_manager.cc:1428] T 62872effdc9a444cafc2fba4fa1bb46f P 8a75da29ae8046b7997d7e7ece52e36b: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 02:40:32.293330 16789 heartbeater.cc:503] Master 127.5.241.126:37859 was elected leader, sending a full tablet report...
I20260619 02:40:32.293128 16806 ts_tablet_manager.cc:1428] T 62872effdc9a444cafc2fba4fa1bb46f P eb298bf474a842dc83c5bed13d2ae53f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260619 02:40:32.315778 16656 tablet.cc:2369] T 62872effdc9a444cafc2fba4fa1bb46f P eb298bf474a842dc83c5bed13d2ae53f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:40:32.414664 16522 tablet.cc:2369] T 62872effdc9a444cafc2fba4fa1bb46f P 301da70e9f6c4b929b03b669a15187f1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:40:32.498207 16790 tablet.cc:2369] T 62872effdc9a444cafc2fba4fa1bb46f P 8a75da29ae8046b7997d7e7ece52e36b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:40:32.712931 16812 raft_consensus.cc:482] T 62872effdc9a444cafc2fba4fa1bb46f P 301da70e9f6c4b929b03b669a15187f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:40:32.713109 16812 raft_consensus.cc:504] T 62872effdc9a444cafc2fba4fa1bb46f P 301da70e9f6c4b929b03b669a15187f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301da70e9f6c4b929b03b669a15187f1" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 45817 } } peers { permanent_uuid: "eb298bf474a842dc83c5bed13d2ae53f" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 45635 } } peers { permanent_uuid: "8a75da29ae8046b7997d7e7ece52e36b" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 46321 } }
I20260619 02:40:32.713464 16812 leader_election.cc:288] T 62872effdc9a444cafc2fba4fa1bb46f P 301da70e9f6c4b929b03b669a15187f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb298bf474a842dc83c5bed13d2ae53f (127.5.241.66:45635), 8a75da29ae8046b7997d7e7ece52e36b (127.5.241.67:46321)
I20260619 02:40:32.717661 16805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:32.713712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:46213 (local address 127.5.241.66:45635)
0619 02:40:32.713764 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:32.713767 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:32.713928 (+ 161us) server_negotiation.cc:407] Connection header received
0619 02:40:32.713966 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:32.713967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:32.713978 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:32.714049 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:40:32.714271 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:32.715043 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:32.716616 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:32.717174 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:32.717262 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:32.717270 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:32.717544 (+ 274us) server_negotiation.cc:299] Negotiation successful
0619 02:40:32.717588 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:40:32.717661 16803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:32.713705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:43603 (local address 127.5.241.67:46321)
0619 02:40:32.713753 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:32.713756 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:32.713928 (+ 172us) server_negotiation.cc:407] Connection header received
0619 02:40:32.713966 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:32.713967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:32.713978 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:32.714049 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:40:32.714271 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:32.715042 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:32.716616 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:32.717194 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:32.717262 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:32.717270 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:32.717544 (+ 274us) server_negotiation.cc:299] Negotiation successful
0619 02:40:32.717588 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 02:40:32.717654 16815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:32.713669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.67:46321 (local address 127.5.241.65:43603)
0619 02:40:32.713829 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:32.713839 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:32.713897 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:32.714067 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:32.714068 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:32.714071 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:40:32.714200 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:32.714201 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:32.715145 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:32.715146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:32.716500 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:32.716504 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:32.717296 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:32.717297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:32.717391 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:32.717398 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 02:40:32.717470 (+ 72us) client_negotiation.cc:240] Negotiation successful
0619 02:40:32.717532 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":82,"spinlock_wait_cycles":10112,"thread_start_us":58,"threads_started":1}
I20260619 02:40:32.717654 16816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:32.713657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.66:45635 (local address 127.5.241.65:46213)
0619 02:40:32.713804 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:32.713815 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:32.713897 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:32.714067 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:32.714068 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:32.714071 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:40:32.714188 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:32.714191 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:32.715145 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:32.715146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:32.716501 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:32.716504 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:32.717296 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:32.717297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:32.717391 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:32.717400 (+ 9us) client_negotiation.cc:769] Sending connection context
0619 02:40:32.717470 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 02:40:32.717532 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":129,"thread_start_us":78,"threads_started":1}
I20260619 02:40:32.718295 16609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62872effdc9a444cafc2fba4fa1bb46f" candidate_uuid: "301da70e9f6c4b929b03b669a15187f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb298bf474a842dc83c5bed13d2ae53f" is_pre_election: true
I20260619 02:40:32.718277 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62872effdc9a444cafc2fba4fa1bb46f" candidate_uuid: "301da70e9f6c4b929b03b669a15187f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a75da29ae8046b7997d7e7ece52e36b" is_pre_election: true
I20260619 02:40:32.718456 16609 raft_consensus.cc:2454] T 62872effdc9a444cafc2fba4fa1bb46f P eb298bf474a842dc83c5bed13d2ae53f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 301da70e9f6c4b929b03b669a15187f1 in term 0.
I20260619 02:40:32.718479 16743 raft_consensus.cc:2454] T 62872effdc9a444cafc2fba4fa1bb46f P 8a75da29ae8046b7997d7e7ece52e36b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 301da70e9f6c4b929b03b669a15187f1 in term 0.
I20260619 02:40:32.718681 16409 leader_election.cc:302] T 62872effdc9a444cafc2fba4fa1bb46f P 301da70e9f6c4b929b03b669a15187f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 301da70e9f6c4b929b03b669a15187f1, eb298bf474a842dc83c5bed13d2ae53f; no voters:
I20260619 02:40:32.718827 16812 raft_consensus.cc:2781] T 62872effdc9a444cafc2fba4fa1bb46f P 301da70e9f6c4b929b03b669a15187f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:40:32.718879 16812 raft_consensus.cc:482] T 62872effdc9a444cafc2fba4fa1bb46f P 301da70e9f6c4b929b03b669a15187f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:40:32.718914 16812 raft_consensus.cc:3037] T 62872effdc9a444cafc2fba4fa1bb46f P 301da70e9f6c4b929b03b669a15187f1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:32.719592 16812 raft_consensus.cc:504] T 62872effdc9a444cafc2fba4fa1bb46f P 301da70e9f6c4b929b03b669a15187f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301da70e9f6c4b929b03b669a15187f1" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 45817 } } peers { permanent_uuid: "eb298bf474a842dc83c5bed13d2ae53f" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 45635 } } peers { permanent_uuid: "8a75da29ae8046b7997d7e7ece52e36b" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 46321 } }
I20260619 02:40:32.719738 16812 leader_election.cc:288] T 62872effdc9a444cafc2fba4fa1bb46f P 301da70e9f6c4b929b03b669a15187f1 [CANDIDATE]: Term 1 election: Requested vote from peers eb298bf474a842dc83c5bed13d2ae53f (127.5.241.66:45635), 8a75da29ae8046b7997d7e7ece52e36b (127.5.241.67:46321)
I20260619 02:40:32.719906 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62872effdc9a444cafc2fba4fa1bb46f" candidate_uuid: "301da70e9f6c4b929b03b669a15187f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a75da29ae8046b7997d7e7ece52e36b"
I20260619 02:40:32.719906 16609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62872effdc9a444cafc2fba4fa1bb46f" candidate_uuid: "301da70e9f6c4b929b03b669a15187f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb298bf474a842dc83c5bed13d2ae53f"
I20260619 02:40:32.719995 16743 raft_consensus.cc:3037] T 62872effdc9a444cafc2fba4fa1bb46f P 8a75da29ae8046b7997d7e7ece52e36b [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:32.719995 16609 raft_consensus.cc:3037] T 62872effdc9a444cafc2fba4fa1bb46f P eb298bf474a842dc83c5bed13d2ae53f [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:32.720875 16609 raft_consensus.cc:2454] T 62872effdc9a444cafc2fba4fa1bb46f P eb298bf474a842dc83c5bed13d2ae53f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 301da70e9f6c4b929b03b669a15187f1 in term 1.
I20260619 02:40:32.720875 16743 raft_consensus.cc:2454] T 62872effdc9a444cafc2fba4fa1bb46f P 8a75da29ae8046b7997d7e7ece52e36b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 301da70e9f6c4b929b03b669a15187f1 in term 1.
I20260619 02:40:32.721036 16409 leader_election.cc:302] T 62872effdc9a444cafc2fba4fa1bb46f P 301da70e9f6c4b929b03b669a15187f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 301da70e9f6c4b929b03b669a15187f1, eb298bf474a842dc83c5bed13d2ae53f; no voters:
I20260619 02:40:32.721140 16812 raft_consensus.cc:2781] T 62872effdc9a444cafc2fba4fa1bb46f P 301da70e9f6c4b929b03b669a15187f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:40:32.721300 16812 raft_consensus.cc:686] T 62872effdc9a444cafc2fba4fa1bb46f P 301da70e9f6c4b929b03b669a15187f1 [term 1 LEADER]: Becoming Leader. State: Replica: 301da70e9f6c4b929b03b669a15187f1, State: Running, Role: LEADER
I20260619 02:40:32.721400 16812 consensus_queue.cc:237] T 62872effdc9a444cafc2fba4fa1bb46f P 301da70e9f6c4b929b03b669a15187f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301da70e9f6c4b929b03b669a15187f1" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 45817 } } peers { permanent_uuid: "eb298bf474a842dc83c5bed13d2ae53f" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 45635 } } peers { permanent_uuid: "8a75da29ae8046b7997d7e7ece52e36b" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 46321 } }
I20260619 02:40:32.722325 16328 catalog_manager.cc:5495] T 62872effdc9a444cafc2fba4fa1bb46f P 301da70e9f6c4b929b03b669a15187f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 301da70e9f6c4b929b03b669a15187f1 (127.5.241.65). New cstate: current_term: 1 leader_uuid: "301da70e9f6c4b929b03b669a15187f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301da70e9f6c4b929b03b669a15187f1" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 45817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb298bf474a842dc83c5bed13d2ae53f" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 45635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a75da29ae8046b7997d7e7ece52e36b" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 46321 } health_report { overall_health: UNKNOWN } } }
I20260619 02:40:32.835186 16821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:32.832542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55034 (local address 127.5.241.65:45817)
0619 02:40:32.832683 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:32.832685 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:32.832695 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:40:32.832789 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:32.832795 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:32.832814 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:32.832897 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:40:32.833057 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:32.833552 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:32.834366 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:32.834562 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:32.834642 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:32.834820 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:40:32.834936 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:40:32.835029 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:32.835068 (+ 39us) server_negotiation.cc:299] Negotiation successful
0619 02:40:32.835090 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":57,"threads_started":1}
I20260619 02:40:32.836910 16743 raft_consensus.cc:1261] T 62872effdc9a444cafc2fba4fa1bb46f P 8a75da29ae8046b7997d7e7ece52e36b [term 1 FOLLOWER]: Refusing update from remote peer 301da70e9f6c4b929b03b669a15187f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:40:32.836910 16609 raft_consensus.cc:1261] T 62872effdc9a444cafc2fba4fa1bb46f P eb298bf474a842dc83c5bed13d2ae53f [term 1 FOLLOWER]: Refusing update from remote peer 301da70e9f6c4b929b03b669a15187f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:40:32.837275 16817 consensus_queue.cc:1035] T 62872effdc9a444cafc2fba4fa1bb46f P 301da70e9f6c4b929b03b669a15187f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb298bf474a842dc83c5bed13d2ae53f" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 45635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:40:32.837462 16812 consensus_queue.cc:1035] T 62872effdc9a444cafc2fba4fa1bb46f P 301da70e9f6c4b929b03b669a15187f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a75da29ae8046b7997d7e7ece52e36b" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 46321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:40:32.841862 16329 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37132:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:40:32.841979 16329 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37132:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:40:32.842759 16329 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ecdb0f02f4c44ecfb00238ec0b01eff2: Sending DeleteTablet for 3 replicas of tablet 62872effdc9a444cafc2fba4fa1bb46f
I20260619 02:40:32.843080 16589 tablet_service.cc:1514] Processing DeleteTablet for tablet 62872effdc9a444cafc2fba4fa1bb46f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:40:32 UTC) from {username='kudu', principal='kudu/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:36250
I20260619 02:40:32.843065 16455 tablet_service.cc:1514] Processing DeleteTablet for tablet 62872effdc9a444cafc2fba4fa1bb46f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:40:32 UTC) from {username='kudu', principal='kudu/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:55022
I20260619 02:40:32.843079 16723 tablet_service.cc:1514] Processing DeleteTablet for tablet 62872effdc9a444cafc2fba4fa1bb46f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:40:32 UTC) from {username='kudu', principal='kudu/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:42008
I20260619 02:40:32.843286 16836 tablet_replica.cc:330] T 62872effdc9a444cafc2fba4fa1bb46f P 8a75da29ae8046b7997d7e7ece52e36b: stopping tablet replica
I20260619 02:40:32.843385 16836 raft_consensus.cc:2229] T 62872effdc9a444cafc2fba4fa1bb46f P 8a75da29ae8046b7997d7e7ece52e36b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:40:32.843490 16836 raft_consensus.cc:2258] T 62872effdc9a444cafc2fba4fa1bb46f P 8a75da29ae8046b7997d7e7ece52e36b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:40:32.843564 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 16392
I20260619 02:40:32.845229 16836 ts_tablet_manager.cc:1905] T 62872effdc9a444cafc2fba4fa1bb46f P 8a75da29ae8046b7997d7e7ece52e36b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:40:32.846509 16834 tablet_replica.cc:330] T 62872effdc9a444cafc2fba4fa1bb46f P eb298bf474a842dc83c5bed13d2ae53f: stopping tablet replica
I20260619 02:40:32.846625 16834 raft_consensus.cc:2229] T 62872effdc9a444cafc2fba4fa1bb46f P eb298bf474a842dc83c5bed13d2ae53f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:40:32.846726 16834 raft_consensus.cc:2258] T 62872effdc9a444cafc2fba4fa1bb46f P eb298bf474a842dc83c5bed13d2ae53f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:40:32.846899 16836 ts_tablet_manager.cc:1918] T 62872effdc9a444cafc2fba4fa1bb46f P 8a75da29ae8046b7997d7e7ece52e36b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:40:32.847008 16836 log.cc:1196] T 62872effdc9a444cafc2fba4fa1bb46f P 8a75da29ae8046b7997d7e7ece52e36b: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/ts-2/wal/wals/62872effdc9a444cafc2fba4fa1bb46f
I20260619 02:40:32.847097 16834 ts_tablet_manager.cc:1905] T 62872effdc9a444cafc2fba4fa1bb46f P eb298bf474a842dc83c5bed13d2ae53f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:40:32.847301 16836 ts_tablet_manager.cc:1939] T 62872effdc9a444cafc2fba4fa1bb46f P 8a75da29ae8046b7997d7e7ece52e36b: Deleting consensus metadata
I20260619 02:40:32.847762 16314 catalog_manager.cc:4841] TS 8a75da29ae8046b7997d7e7ece52e36b (127.5.241.67:46321): tablet 62872effdc9a444cafc2fba4fa1bb46f (table test-table [id=4db4dfb588b242d29f3653d01024e9b4]) successfully deleted
I20260619 02:40:32.848613 16834 ts_tablet_manager.cc:1918] T 62872effdc9a444cafc2fba4fa1bb46f P eb298bf474a842dc83c5bed13d2ae53f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:40:32.848698 16834 log.cc:1196] T 62872effdc9a444cafc2fba4fa1bb46f P eb298bf474a842dc83c5bed13d2ae53f: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836772852275-6085-0/minicluster-data/ts-1/wal/wals/62872effdc9a444cafc2fba4fa1bb46f
W20260619 02:40:32.848927 16316 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.241.65:45817 (error 108)
I20260619 02:40:32.848955 16834 ts_tablet_manager.cc:1939] T 62872effdc9a444cafc2fba4fa1bb46f P eb298bf474a842dc83c5bed13d2ae53f: Deleting consensus metadata
I20260619 02:40:32.849185 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 16526
I20260619 02:40:32.849830 16838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:32.849252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:45817 (local address 127.0.0.1:55038)
0619 02:40:32.849742 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:32.849782 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.65:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":38,"threads_started":1}
W20260619 02:40:32.849934 16316 catalog_manager.cc:4584] TS 301da70e9f6c4b929b03b669a15187f1 (127.5.241.65:45817): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 62872effdc9a444cafc2fba4fa1bb46f: Network error: Client connection negotiation failed: client connection to 127.5.241.65:45817: connect: Connection refused (error 111)
I20260619 02:40:32.853890 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 16660
I20260619 02:40:32.853935 16838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:32.853819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.66:45635 (local address 127.0.0.1:36258)
0619 02:40:32.853875 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:32.853900 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.66:45635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260619 02:40:32.858246 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 16298
2026-06-19T02:40:32Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2439 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:40:32 dist-test-slave-zpbg krb5kdc[16841](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:40:32 dist-test-slave-zpbg krb5kdc[16841](info): set up 2 sockets
Jun 19 02:40:32 dist-test-slave-zpbg krb5kdc[16841](info): commencing operation
krb5kdc: starting...
W20260619 02:40:33.903764 6085 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Jun 19 02:40:33 dist-test-slave-zpbg krb5kdc[16841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836833, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:40:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:40:33Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.5.241.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.5.241.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.5.241.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/krb5kdc/rangeradmin_127.5.241.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.5.241.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/krb5kdc/rangeradmin_127.5.241.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.5.241.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.5.241.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.5.241.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/krb5kdc/rangerlookup_127.5.241.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.5.241.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/krb5kdc/rangerlookup_127.5.241.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.241.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.5.241.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.241.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/krb5kdc/HTTP_127.5.241.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.241.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/krb5kdc/HTTP_127.5.241.84@KRBTEST.COM.keytab.
I20260619 02:40:34.067373 6085 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskflgrG6/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/postgres -l logfile start
2026-06-19 02:40:36.651 UTC [16871] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-19 02:40:36.651 UTC [16871] LOG: listening on IPv4 address "127.5.241.84", port 33417
2026-06-19 02:40:36.656 UTC [16871] LOG: listening on Unix socket "/tmp/.s.PGSQL.33417"
2026-06-19 02:40:36.679 UTC [16874] LOG: database system was shut down at 2026-06-19 02:40:35 UTC
2026-06-19 02:40:36.683 UTC [16871] LOG: database system is ready to accept connections
I20260619 02:40:37.627823 6085 mini_postgres.cc:96] Postgres bound to 33417
2026-06-19 02:40:37.636 UTC [16882] FATAL: database "slave" does not exist
127.5.241.84:33417 - accepting connections
I20260619 02:40:37.636584 6085 mini_ranger.cc:161] Starting Ranger...
I20260619 02:40:37.651028 6085 mini_ranger.cc:84] Created miniranger Postgres user
I20260619 02:40:38.156121 6085 mini_ranger.cc:87] Created ranger Postgres database
I20260619 02:40:38.156190 6085 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ranger-admin
2026-06-19 02:40:38,439 [I] DB FLAVOR :POSTGRES
2026-06-19 02:40:38,439 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 02:40:38,440 [I] Checking connection..
2026-06-19 02:40:38,440 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskflgrG6/build/release/bin/postgresql.jar:/tmp/dist-test-taskflgrG6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.241.84:33417/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-19 02:40:38,708 [I] Checking connection passed.
2026-06-19 02:40:38,708 [I] --------- Verifying version history table ---------
2026-06-19 02:40:38,709 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskflgrG6/build/release/bin/postgresql.jar:/tmp/dist-test-taskflgrG6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.241.84:33417/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 02:40:38,968 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 02:40:38,968 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskflgrG6/build/release/bin/postgresql.jar:/tmp/dist-test-taskflgrG6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.241.84:33417/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 02:40:39,202 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 02:40:39,202 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-19 02:40:39,202 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskflgrG6/build/release/bin/postgresql.jar:/tmp/dist-test-taskflgrG6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.241.84:33417/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskflgrG6/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-19 02:40:39.459 UTC [16946] WARNING: there is no transaction in progress
2026-06-19 02:40:39,465 [I] create_dbversion_catalog.sql file imported successfully
2026-06-19 02:40:39,466 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskflgrG6/build/release/bin/postgresql.jar:/tmp/dist-test-taskflgrG6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.241.84:33417/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 02:40:39,708 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-19 02:40:39,708 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 02:40:39,709 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskflgrG6/build/release/bin/postgresql.jar:/tmp/dist-test-taskflgrG6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.241.84:33417/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-19 02:40:39,940 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskflgrG6/build/release/bin/postgresql.jar:/tmp/dist-test-taskflgrG6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.241.84:33417/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-19 02:40:40,186 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskflgrG6/build/release/bin/postgresql.jar:/tmp/dist-test-taskflgrG6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.241.84:33417/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-zpbg.c.gcp-upstream.internal','N') ;"
2026-06-19 02:40:40,437 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskflgrG6/build/release/bin/postgresql.jar:/tmp/dist-test-taskflgrG6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.241.84:33417/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 02:40:40,689 [I] Table x_portal_user does not exist in database ranger
2026-06-19 02:40:40,689 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskflgrG6/build/release/bin/postgresql.jar:/tmp/dist-test-taskflgrG6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.241.84:33417/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 02:40:40,925 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-19 02:40:40,925 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskflgrG6/build/release/bin/postgresql.jar:/tmp/dist-test-taskflgrG6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.241.84:33417/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 02:40:41,151 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskflgrG6/build/release/bin/postgresql.jar:/tmp/dist-test-taskflgrG6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.241.84:33417/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 02:40:41,381 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-19 02:40:41,381 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskflgrG6/build/release/bin/postgresql.jar:/tmp/dist-test-taskflgrG6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.241.84:33417/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskflgrG6/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-19 02:40:42.372 UTC [17054] WARNING: there is no transaction in progress
2026-06-19 02:40:42.389 UTC [17054] WARNING: there is no transaction in progress
2026-06-19 02:40:42.406 UTC [17054] WARNING: there is no transaction in progress
2026-06-19 02:40:42.422 UTC [17054] WARNING: there is no transaction in progress
2026-06-19 02:40:42.438 UTC [17054] WARNING: there is no transaction in progress
2026-06-19 02:40:42.456 UTC [17054] WARNING: there is no transaction in progress
2026-06-19 02:40:42.509 UTC [17054] WARNING: there is no transaction in progress
2026-06-19 02:40:42.517 UTC [17054] WARNING: there is no transaction in progress
2026-06-19 02:40:42.532 UTC [17054] WARNING: there is no transaction in progress
2026-06-19 02:40:42.542 UTC [17054] WARNING: there is no transaction in progress
2026-06-19 02:40:42.552 UTC [17054] WARNING: there is no transaction in progress
2026-06-19 02:40:42.566 UTC [17054] WARNING: there is no transaction in progress
2026-06-19 02:40:42.575 UTC [17054] WARNING: there is no transaction in progress
2026-06-19 02:40:42.582 UTC [17054] WARNING: there is no transaction in progress
2026-06-19 02:40:42.589 UTC [17054] WARNING: there is no transaction in progress
2026-06-19 02:40:42.596 UTC [17054] WARNING: there is no transaction in progress
2026-06-19 02:40:43,267 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-19 02:40:43,267 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskflgrG6/build/release/bin/postgresql.jar:/tmp/dist-test-taskflgrG6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.241.84:33417/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-19 02:40:43,509 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-19 02:40:43,510 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskflgrG6/build/release/bin/postgresql.jar:/tmp/dist-test-taskflgrG6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.241.84:33417/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 02:40:43,759 [I] Table x_portal_user already exists in database 'ranger'
2026-06-19 02:40:43,759 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskflgrG6/build/release/bin/postgresql.jar:/tmp/dist-test-taskflgrG6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.241.84:33417/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 02:40:44,024 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-19 02:40:44,024 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskflgrG6/build/release/bin/postgresql.jar:/tmp/dist-test-taskflgrG6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.241.84:33417/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 02:40:44,284 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskflgrG6/build/release/bin/postgresql.jar:/tmp/dist-test-taskflgrG6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.241.84:33417/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 02:40:44,516 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskflgrG6/build/release/bin/postgresql.jar:/tmp/dist-test-taskflgrG6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.241.84:33417/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-zpbg.c.gcp-upstream.internal';"
2026-06-19 02:40:44,745 [I] CORE_DB_SCHEMA import status has been updated
2026-06-19 02:40:44,745 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskflgrG6/build/release/bin/postgresql.jar:/tmp/dist-test-taskflgrG6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.241.84:33417/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-19 02:40:44,971 [I] DB_PATCHES have already been applied
I20260619 02:40:44.977438 6085 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ranger-admin:/tmp/dist-test-taskflgrG6/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskflgrG6/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskflgrG6/build/release/bin/postgresql.jar:/tmp/dist-test-taskflgrG6/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260619 02:40:44.977505 6085 mini_ranger.cc:193] Using host: 127.5.241.84
I20260619 02:40:44.980571 6085 mini_ranger.cc:227] Ranger admin URL: http://127.5.241.84:38911
Jun 19, 2026 2:40:45 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 2:40:45 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 2:40:45 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ranger-admin/ews/webapp
Jun 19, 2026 2:40:45 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 19, 2026 2:40:45 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 19, 2026 2:40:45 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ranger-admin/ews/webapp].
Jun 19, 2026 2:40:45 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260619 02:40:46.077159 6085 mini_ranger.cc:160] Time spent starting Ranger: real 8.441s user 0.003s sys 0.006s
Jun 19, 2026 2:41:06 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 19, 2026 2:41:06 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Jun 19, 2026 2:41:06 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 19, 2026 2:41:06 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 19, 2026 2:41:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 19, 2026 2:41:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 19, 2026 2:41:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 19, 2026 2:41:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 19, 2026 2:41:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 19, 2026 2:41:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 19, 2026 2:41:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 19, 2026 2:41:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 19, 2026 2:41:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 19, 2026 2:41:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 19, 2026 2:41:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 19, 2026 2:41:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 19, 2026 2:41:06 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 19, 2026 2:41:07 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260619 02:41:08.094643 6085 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.5.241.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.5.241.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.5.241.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/krb5kdc/rangerkms_127.5.241.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.5.241.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/krb5kdc/rangerkms_127.5.241.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.241.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.5.241.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.241.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/krb5kdc/HTTP_127.5.241.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.241.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/krb5kdc/HTTP_127.5.241.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 19 02:41:08 dist-test-slave-zpbg krb5kdc[16841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836868, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260619 02:41:08.183260 6085 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260619 02:41:08.193701 6085 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260619 02:41:09.054425 6085 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260619 02:41:09.054505 6085 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ranger-kms
I20260619 02:41:09.054528 6085 mini_ranger_kms.cc:227] Using postgres at 127.5.241.84:33417
2026-06-19 02:41:09,236 [I] DB FLAVOR :POSTGRES
2026-06-19 02:41:09,236 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 02:41:09,236 [I] Checking connection
2026-06-19 02:41:09,236 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskflgrG6/build/release/bin/postgresql.jar:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.241.84:33417/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-19 02:41:09,472 [I] connection success
2026-06-19 02:41:09,473 [I] --------- Verifying Ranger DB tables ---------
2026-06-19 02:41:09,473 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-19 02:41:09,473 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskflgrG6/build/release/bin/postgresql.jar:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.241.84:33417/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-19 02:41:09,721 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-19 02:41:09,721 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 02:41:09,722 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-19 02:41:09,722 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskflgrG6/build/release/bin/postgresql.jar:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.241.84:33417/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-19 02:41:09,974 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260619 02:41:10.443186 6085 mini_ranger_kms.cc:310] Created kms service
I20260619 02:41:10.650823 6085 mini_ranger_kms.cc:326] Created kudu user
I20260619 02:41:10.695771 6085 mini_ranger_kms.cc:343] Created rangerkms user
Jun 19 02:41:10 dist-test-slave-zpbg krb5kdc[16841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836868, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.5.241.84@KRBTEST.COM
I20260619 02:41:11.109946 6085 mini_ranger_kms.cc:384] Added ranger policy
I20260619 02:41:11.110059 6085 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ranger-kms:/tmp/dist-test-taskflgrG6/build/release/bin/postgresql.jar:/tmp/dist-test-taskflgrG6/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskflgrG6/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260619 02:41:11.110081 6085 mini_ranger_kms.cc:240] Using host: 127.5.241.84
I20260619 02:41:11.113411 6085 mini_ranger_kms.cc:276] Ranger KMS PID: 17234
I20260619 02:41:11.113533 6085 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.5.241.84:52375
Jun 19, 2026 2:41:12 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 2:41:12 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 2:41:12 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 19, 2026 2:41:12 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 19, 2026 2:41:12 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ranger-kms/ews/webapp].
Jun 19, 2026 2:41:12 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260619 02:41:13.338846 6085 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.156s user 0.007s sys 0.157s
I20260619 02:41:13.338960 6085 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260619 02:41:13.339040 6085 mini_ranger_kms.cc:401] 127.5.241.84:52375/kms/v1/keys
Jun 19 02:41:13 dist-test-slave-zpbg krb5kdc[16841](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781836873, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.241.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 19 02:41:13 dist-test-slave-zpbg krb5kdc[16841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836873, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.241.84@KRBTEST.COM for HTTP/127.5.241.84@KRBTEST.COM
Jun 19 02:41:14 dist-test-slave-zpbg krb5kdc[16841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836873, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.241.84@KRBTEST.COM for HTTP/127.5.241.84@KRBTEST.COM
Jun 19 02:41:16 dist-test-slave-zpbg krb5kdc[16841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.5.241.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
I20260619 02:41:16.590507 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:36063
--webserver_interface=127.5.241.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:46445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:36063
--ranger_config_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.241.84:52375/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:41:16.672876 17285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:16.673089 17285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:16.673111 17285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:16.674274 17285 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:41:16.674321 17285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:16.674336 17285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:16.674348 17285 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:41:16.674361 17285 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:41:16.675988 17285 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:46445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/master-0/wal
--ranger_kms_url=127.5.241.84:52375/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:36063
--ranger_config_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:36063
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.241.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17285
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:41:16.676241 17285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:16.676491 17285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:41:16.681787 17285 server_base.cc:1034] running on GCE node
W20260619 02:41:16.681890 17290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:16.681964 17293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:16.682184 17291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:16.682488 17285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:16.682864 17285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:16.684043 17285 hybrid_clock.cc:648] HybridClock initialized: now 1781836876684019 us; error 39 us; skew 500 ppm
Jun 19 02:41:16 dist-test-slave-zpbg krb5kdc[16841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836876, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:41:16.687095 17285 init.cc:377] Logged in from keytab as kudu/127.5.241.126@KRBTEST.COM (short username kudu)
I20260619 02:41:16.687649 17285 webserver.cc:460] Webserver started at http://127.5.241.126:42643/ using document root <none> and password file <none>
I20260619 02:41:16.687889 17285 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:16.687930 17285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:16.688066 17285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 02:41:16 dist-test-slave-zpbg krb5kdc[16841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836876, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for HTTP/127.5.241.84@KRBTEST.COM
I20260619 02:41:16.733040 17285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/master-0/data/instance:
uuid: "798799cac97345efb7c01712070d2d70"
format_stamp: "Formatted at 2026-06-19 02:41:16 on dist-test-slave-zpbg"
server_key: "23ad480b4c87c0586b959998cbd7dc6e"
server_key_iv: "7c5d773efe787fe1139b5b77cec8014b"
server_key_version: "kuduclusterkey@0"
I20260619 02:41:16.733455 17285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/master-0/wal/instance:
uuid: "798799cac97345efb7c01712070d2d70"
format_stamp: "Formatted at 2026-06-19 02:41:16 on dist-test-slave-zpbg"
server_key: "23ad480b4c87c0586b959998cbd7dc6e"
server_key_iv: "7c5d773efe787fe1139b5b77cec8014b"
server_key_version: "kuduclusterkey@0"
I20260619 02:41:16.734967 17285 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 02:41:16.750753 17300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:16.751012 17285 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 02:41:16.751128 17285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "798799cac97345efb7c01712070d2d70"
format_stamp: "Formatted at 2026-06-19 02:41:16 on dist-test-slave-zpbg"
server_key: "23ad480b4c87c0586b959998cbd7dc6e"
server_key_iv: "7c5d773efe787fe1139b5b77cec8014b"
server_key_version: "kuduclusterkey@0"
I20260619 02:41:16.751233 17285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:16.779853 17285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:16.781962 17285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:16.782119 17285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:16.787345 17285 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.126:36063
I20260619 02:41:16.787372 17352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.126:36063 every 8 connection(s)
I20260619 02:41:16.787734 17285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:41:16.788504 17353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:41:16.791918 17353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 798799cac97345efb7c01712070d2d70: Bootstrap starting.
I20260619 02:41:16.792613 17353 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 798799cac97345efb7c01712070d2d70: Neither blocks nor log segments found. Creating new log.
I20260619 02:41:16.792902 17353 log.cc:824] T 00000000000000000000000000000000 P 798799cac97345efb7c01712070d2d70: Log is configured to *not* fsync() on all Append() calls
I20260619 02:41:16.793664 17353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 798799cac97345efb7c01712070d2d70: No bootstrap required, opened a new log
I20260619 02:41:16.796397 17353 raft_consensus.cc:348] T 00000000000000000000000000000000 P 798799cac97345efb7c01712070d2d70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "798799cac97345efb7c01712070d2d70" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 36063 } }
I20260619 02:41:16.796583 17353 raft_consensus.cc:374] T 00000000000000000000000000000000 P 798799cac97345efb7c01712070d2d70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:41:16.796557 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 17285
I20260619 02:41:16.796615 17353 raft_consensus.cc:729] T 00000000000000000000000000000000 P 798799cac97345efb7c01712070d2d70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 798799cac97345efb7c01712070d2d70, State: Initialized, Role: FOLLOWER
I20260619 02:41:16.796679 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/master-0/wal/instance
I20260619 02:41:16.796784 17353 consensus_queue.cc:260] T 00000000000000000000000000000000 P 798799cac97345efb7c01712070d2d70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "798799cac97345efb7c01712070d2d70" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 36063 } }
I20260619 02:41:16.796849 17353 raft_consensus.cc:388] T 00000000000000000000000000000000 P 798799cac97345efb7c01712070d2d70 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:41:16.796875 17353 raft_consensus.cc:482] T 00000000000000000000000000000000 P 798799cac97345efb7c01712070d2d70 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:41:16.796868 6085 external_mini_cluster.cc:1428] Setting key 0987622166adea7241bfb3b2e1fdf644
I20260619 02:41:16.796939 17353 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 798799cac97345efb7c01712070d2d70 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:41:16.797611 17353 raft_consensus.cc:504] T 00000000000000000000000000000000 P 798799cac97345efb7c01712070d2d70 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "798799cac97345efb7c01712070d2d70" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 36063 } }
I20260619 02:41:16.797715 17353 leader_election.cc:302] T 00000000000000000000000000000000 P 798799cac97345efb7c01712070d2d70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 798799cac97345efb7c01712070d2d70; no voters:
I20260619 02:41:16.797902 17353 leader_election.cc:288] T 00000000000000000000000000000000 P 798799cac97345efb7c01712070d2d70 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:41:16.797956 17358 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 798799cac97345efb7c01712070d2d70 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:41:16.798178 17358 raft_consensus.cc:686] T 00000000000000000000000000000000 P 798799cac97345efb7c01712070d2d70 [term 1 LEADER]: Becoming Leader. State: Replica: 798799cac97345efb7c01712070d2d70, State: Running, Role: LEADER
I20260619 02:41:16.798205 17353 sys_catalog.cc:564] T 00000000000000000000000000000000 P 798799cac97345efb7c01712070d2d70 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:41:16.798341 17358 consensus_queue.cc:237] T 00000000000000000000000000000000 P 798799cac97345efb7c01712070d2d70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "798799cac97345efb7c01712070d2d70" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 36063 } }
I20260619 02:41:16.798669 17358 sys_catalog.cc:455] T 00000000000000000000000000000000 P 798799cac97345efb7c01712070d2d70 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 798799cac97345efb7c01712070d2d70. Latest consensus state: current_term: 1 leader_uuid: "798799cac97345efb7c01712070d2d70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "798799cac97345efb7c01712070d2d70" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 36063 } } }
I20260619 02:41:16.798733 17358 sys_catalog.cc:458] T 00000000000000000000000000000000 P 798799cac97345efb7c01712070d2d70 [sys.catalog]: This master's current role is: LEADER
I20260619 02:41:16.798949 17365 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:41:16.799415 17365 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:41:16.799784 17353 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 02:41:16.800045 17359 sys_catalog.cc:455] T 00000000000000000000000000000000 P 798799cac97345efb7c01712070d2d70 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "798799cac97345efb7c01712070d2d70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "798799cac97345efb7c01712070d2d70" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 36063 } } }
I20260619 02:41:16.800263 17359 sys_catalog.cc:458] T 00000000000000000000000000000000 P 798799cac97345efb7c01712070d2d70 [sys.catalog]: This master's current role is: LEADER
I20260619 02:41:16.805323 17375 mvcc.cc:205] Tried to move back new op lower bound from 7298403847385333760 to 7298403847366311936. Current Snapshot: MvccSnapshot[applied={T|T < 7298403847385333760}]
Jun 19 02:41:16 dist-test-slave-zpbg krb5kdc[16841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:41:16.807387 17365 catalog_manager.cc:1347] Generated new cluster ID: 348cd50e1c064cda887394ed58e237e4
I20260619 02:41:16.807458 17365 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:41:16.812551 17357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:16.797375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36486 (local address 127.5.241.126:36063)
0619 02:41:16.797549 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:16.797558 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:16.797590 (+ 32us) server_negotiation.cc:407] Connection header received
0619 02:41:16.797761 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:16.797769 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:16.797829 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:16.797991 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:16.798926 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:16.800326 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:16.802101 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:16.803817 (+ 1716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:16.804425 (+ 608us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:16.807631 (+ 3206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:16.807636 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:16.807636 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:16.807648 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:16.810606 (+ 2958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:16.810992 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:16.810995 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:16.810995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:16.811099 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:16.811285 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:16.811286 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:16.811286 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:16.811576 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:16.812082 (+ 506us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:16.812161 (+ 79us) server_negotiation.cc:299] Negotiation successful
0619 02:41:16.812305 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":73,"threads_started":1}
I20260619 02:41:16.832473 17365 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:41:16.833321 17365 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:41:16.838798 17365 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 798799cac97345efb7c01712070d2d70: Generated new TSK 0
I20260619 02:41:16.839058 17365 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.241.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab.
Entry for principal kudu/127.5.241.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab.
I20260619 02:41:17.623276 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.241.84:52375/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.65:0
--local_ip_for_outbound_sockets=127.5.241.65
--webserver_interface=127.5.241.65
--webserver_port=0
--tserver_master_addrs=127.5.241.126:36063
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab
--principal=kudu/127.5.241.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:46445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:41:17.752780 17398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:17.753144 17398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:17.753232 17398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:17.755139 17398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:17.755210 17398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:17.755254 17398 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.65
I20260619 02:41:17.757499 17398 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:46445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.5.241.84:52375/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab
--principal=kudu/127.5.241.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.241.65
--webserver_port=0
--tserver_master_addrs=127.5.241.126:36063
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17398
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.65
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:41:17.757755 17398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:17.758060 17398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:17.764882 17406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:17.765196 17403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:17.765386 17398 server_base.cc:1034] running on GCE node
W20260619 02:41:17.765394 17404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:17.765738 17398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:17.766003 17398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:17.767252 17398 hybrid_clock.cc:648] HybridClock initialized: now 1781836877767229 us; error 34 us; skew 500 ppm
Jun 19 02:41:17 dist-test-slave-zpbg krb5kdc[16841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836877, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:41:17.770604 17398 init.cc:377] Logged in from keytab as kudu/127.5.241.65@KRBTEST.COM (short username kudu)
I20260619 02:41:17.771121 17398 webserver.cc:460] Webserver started at http://127.5.241.65:37503/ using document root <none> and password file <none>
I20260619 02:41:17.771385 17398 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:17.771458 17398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:17.771600 17398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 02:41:17 dist-test-slave-zpbg krb5kdc[16841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836877, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.65@KRBTEST.COM for HTTP/127.5.241.84@KRBTEST.COM
I20260619 02:41:17.785929 17398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ts-0/data/instance:
uuid: "9bbd3bc6d19647e1b487a2b123cabe6a"
format_stamp: "Formatted at 2026-06-19 02:41:17 on dist-test-slave-zpbg"
server_key: "2e43db2f0e3668d583d98cb3d5613b84"
server_key_iv: "254f5c1b402d191aa019b0a7eab56aed"
server_key_version: "kuduclusterkey@0"
I20260619 02:41:17.786473 17398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ts-0/wal/instance:
uuid: "9bbd3bc6d19647e1b487a2b123cabe6a"
format_stamp: "Formatted at 2026-06-19 02:41:17 on dist-test-slave-zpbg"
server_key: "2e43db2f0e3668d583d98cb3d5613b84"
server_key_iv: "254f5c1b402d191aa019b0a7eab56aed"
server_key_version: "kuduclusterkey@0"
I20260619 02:41:17.791182 17398 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260619 02:41:17.804244 17414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:17.804490 17398 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:41:17.804646 17398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ts-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ts-0/wal
uuid: "9bbd3bc6d19647e1b487a2b123cabe6a"
format_stamp: "Formatted at 2026-06-19 02:41:17 on dist-test-slave-zpbg"
server_key: "2e43db2f0e3668d583d98cb3d5613b84"
server_key_iv: "254f5c1b402d191aa019b0a7eab56aed"
server_key_version: "kuduclusterkey@0"
I20260619 02:41:17.804754 17398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:17.820345 17398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:17.822716 17398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:17.822938 17398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:17.823242 17398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:41:17.823639 17398 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:41:17.823722 17398 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:17.823817 17398 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:41:17.823885 17398 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:17.830506 17398 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.65:44921
I20260619 02:41:17.831025 17398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
I20260619 02:41:17.833065 17528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.65:44921 every 8 connection(s)
I20260619 02:41:17.835563 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 17398
I20260619 02:41:17.835666 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ts-0/wal/instance
I20260619 02:41:17.835856 6085 external_mini_cluster.cc:1428] Setting key 0469f105241c42ffa9f3a699ff4b11ae
Jun 19 02:41:17 dist-test-slave-zpbg krb5kdc[16841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836877, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.65@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:41:17.851576 17532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:17.832230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:47419 (local address 127.5.241.126:36063)
0619 02:41:17.834207 (+ 1977us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:17.834210 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:17.834218 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:41:17.834246 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:17.834247 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:17.834260 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:17.834335 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:17.835143 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:17.835945 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:17.840406 (+ 4461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:17.840738 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:17.840813 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:17.847860 (+ 7047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:17.847862 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:17.847863 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:17.847872 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:17.849816 (+ 1944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:17.850362 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:17.850364 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:17.850364 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:17.850434 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:17.850751 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:17.850752 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:17.850752 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:17.850940 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:17.851012 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:17.851374 (+ 362us) server_negotiation.cc:299] Negotiation successful
0619 02:41:17.851465 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1943,"thread_start_us":86,"threads_started":1}
I20260619 02:41:17.851758 17531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:17.831926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:36063 (local address 127.5.241.65:47419)
0619 02:41:17.833365 (+ 1439us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:17.833407 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:17.834000 (+ 593us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:17.834457 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:17.834462 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:17.834708 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:17.835019 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:17.835024 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:17.836133 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:17.836134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:17.839548 (+ 3414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:17.839552 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:17.842052 (+ 2500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:17.842053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:17.842219 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:17.842921 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:17.842933 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:17.844713 (+ 1780us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:17.849906 (+ 5193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:17.849915 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:17.849918 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:17.850244 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:17.850495 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:17.850496 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:17.850497 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:17.850653 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:17.851006 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:17.851013 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:17.851195 (+ 182us) client_negotiation.cc:769] Sending connection context
0619 02:41:17.851330 (+ 135us) client_negotiation.cc:240] Negotiation successful
0619 02:41:17.851504 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1395,"thread_start_us":89,"threads_started":1}
I20260619 02:41:17.852831 17529 heartbeater.cc:347] Connected to a master server at 127.5.241.126:36063
I20260619 02:41:17.852939 17529 heartbeater.cc:464] Registering TS with master...
I20260619 02:41:17.853128 17529 heartbeater.cc:511] Master 127.5.241.126:36063 requested a full tablet report, sending...
I20260619 02:41:17.853673 17317 ts_manager.cc:194] Registered new tserver with Master: 9bbd3bc6d19647e1b487a2b123cabe6a (127.5.241.65:44921)
I20260619 02:41:17.854601 17317 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.65@KRBTEST.COM'} at 127.5.241.65:47419
WARNING: no policy specified for kudu/127.5.241.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab.
Entry for principal kudu/127.5.241.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab.
I20260619 02:41:17.875952 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.241.84:52375/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.66:0
--local_ip_for_outbound_sockets=127.5.241.66
--webserver_interface=127.5.241.66
--webserver_port=0
--tserver_master_addrs=127.5.241.126:36063
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab
--principal=kudu/127.5.241.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:46445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:41:17.997540 17535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:17.997941 17535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:17.998014 17535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:17.999927 17535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:18.000056 17535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:18.000119 17535 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.66
Jun 19 02:41:17 dist-test-slave-zpbg krb5kdc[16841](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781836877, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:41:18.002259 17535 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:46445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.5.241.84:52375/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab
--principal=kudu/127.5.241.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.241.66
--webserver_port=0
--tserver_master_addrs=127.5.241.126:36063
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.66
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:41:18.002540 17535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:18.002837 17535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:18.010648 17543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:18.010699 17542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:18.010650 17545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:18.016103 17535 server_base.cc:1034] running on GCE node
I20260619 02:41:18.016495 17535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:18.016767 17535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:18.017913 17535 hybrid_clock.cc:648] HybridClock initialized: now 1781836878017890 us; error 37 us; skew 500 ppm
Jun 19 02:41:18 dist-test-slave-zpbg krb5kdc[16841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836878, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:41:18.021236 17535 init.cc:377] Logged in from keytab as kudu/127.5.241.66@KRBTEST.COM (short username kudu)
I20260619 02:41:18.021788 17535 webserver.cc:460] Webserver started at http://127.5.241.66:33619/ using document root <none> and password file <none>
I20260619 02:41:18.022117 17535 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:18.022233 17535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:18.022444 17535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 02:41:18 dist-test-slave-zpbg krb5kdc[16841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836878, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.66@KRBTEST.COM for HTTP/127.5.241.84@KRBTEST.COM
I20260619 02:41:18.036901 17535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ts-1/data/instance:
uuid: "47bf02c7985044b29cfd995c0552099b"
format_stamp: "Formatted at 2026-06-19 02:41:18 on dist-test-slave-zpbg"
server_key: "c9bd53c0f797e4e7d2cb56bd9e7abb5f"
server_key_iv: "3cb786439a7ab1e56f3b55e44fc76288"
server_key_version: "kuduclusterkey@0"
I20260619 02:41:18.037467 17535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ts-1/wal/instance:
uuid: "47bf02c7985044b29cfd995c0552099b"
format_stamp: "Formatted at 2026-06-19 02:41:18 on dist-test-slave-zpbg"
server_key: "c9bd53c0f797e4e7d2cb56bd9e7abb5f"
server_key_iv: "3cb786439a7ab1e56f3b55e44fc76288"
server_key_version: "kuduclusterkey@0"
I20260619 02:41:18.040031 17535 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260619 02:41:18.058945 17552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:18.059571 17535 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.001s sys 0.000s
I20260619 02:41:18.059703 17535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ts-1/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ts-1/wal
uuid: "47bf02c7985044b29cfd995c0552099b"
format_stamp: "Formatted at 2026-06-19 02:41:18 on dist-test-slave-zpbg"
server_key: "c9bd53c0f797e4e7d2cb56bd9e7abb5f"
server_key_iv: "3cb786439a7ab1e56f3b55e44fc76288"
server_key_version: "kuduclusterkey@0"
I20260619 02:41:18.059815 17535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:18.082159 17535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:18.084483 17535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:18.084682 17535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:18.084959 17535 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:41:18.085745 17535 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:41:18.085800 17535 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:18.085858 17535 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:41:18.085887 17535 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:18.093272 17535 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.66:46269
I20260619 02:41:18.093313 17666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.66:46269 every 8 connection(s)
I20260619 02:41:18.093729 17535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
I20260619 02:41:18.094190 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 17535
I20260619 02:41:18.094301 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ts-1/wal/instance
I20260619 02:41:18.094470 6085 external_mini_cluster.cc:1428] Setting key e39779eaddbdcecdf8e17c97b4509175
WARNING: no policy specified for kudu/127.5.241.67@KRBTEST.COM; defaulting to no policy
Jun 19 02:41:18 dist-test-slave-zpbg krb5kdc[16841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836878, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.66@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:41:18.111876 17532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:18.095194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:44451 (local address 127.5.241.126:36063)
0619 02:41:18.095242 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:18.095244 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:18.095532 (+ 288us) server_negotiation.cc:407] Connection header received
0619 02:41:18.096389 (+ 857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:18.096391 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:18.096408 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:18.096440 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:18.097325 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.098144 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.100799 (+ 2655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.101128 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.101172 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:18.108262 (+ 7090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:18.108265 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:18.108265 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:18.108274 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:18.110134 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:18.110677 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:18.110678 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:18.110679 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:18.110738 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:18.111083 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:18.111084 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:18.111085 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:18.111227 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:18.111270 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:18.111662 (+ 392us) server_negotiation.cc:299] Negotiation successful
0619 02:41:18.111753 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 02:41:18.112651 17669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:18.095169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:36063 (local address 127.5.241.66:44451)
0619 02:41:18.095405 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:18.095450 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:18.096212 (+ 762us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:18.096625 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:18.096630 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:18.096893 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:18.097196 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:18.097203 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.098236 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.098237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:18.099246 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:18.099249 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.101269 (+ 2020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.101270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:18.101428 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:18.106515 (+ 5087us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:18.106526 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:18.108136 (+ 1610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:18.110259 (+ 2123us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:18.110266 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:18.110269 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:18.110579 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:18.110861 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:18.110862 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:18.110862 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:18.111000 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:18.111331 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:18.111337 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:18.111503 (+ 166us) client_negotiation.cc:769] Sending connection context
0619 02:41:18.112231 (+ 728us) client_negotiation.cc:240] Negotiation successful
0619 02:41:18.112420 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":88,"threads_started":1}
I20260619 02:41:18.113312 17667 heartbeater.cc:347] Connected to a master server at 127.5.241.126:36063
I20260619 02:41:18.113407 17667 heartbeater.cc:464] Registering TS with master...
I20260619 02:41:18.113588 17667 heartbeater.cc:511] Master 127.5.241.126:36063 requested a full tablet report, sending...
I20260619 02:41:18.113991 17317 ts_manager.cc:194] Registered new tserver with Master: 47bf02c7985044b29cfd995c0552099b (127.5.241.66:46269)
I20260619 02:41:18.114496 17317 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.66@KRBTEST.COM'} at 127.5.241.66:44451
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab.
Entry for principal kudu/127.5.241.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab.
I20260619 02:41:18.135948 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.241.84:52375/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.67:0
--local_ip_for_outbound_sockets=127.5.241.67
--webserver_interface=127.5.241.67
--webserver_port=0
--tserver_master_addrs=127.5.241.126:36063
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab
--principal=kudu/127.5.241.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:46445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:41:18.253710 17672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:18.254114 17672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:18.254223 17672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:18.256172 17672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:18.256387 17672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:18.256490 17672 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.67
I20260619 02:41:18.258764 17672 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:46445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.5.241.84:52375/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab
--principal=kudu/127.5.241.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.241.67
--webserver_port=0
--tserver_master_addrs=127.5.241.126:36063
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17672
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.67
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:41:18.259254 17672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:18.259572 17672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:18.266247 17678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:18.266248 17677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:18.267495 17672 server_base.cc:1034] running on GCE node
W20260619 02:41:18.271324 17680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:18.271633 17672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:18.271919 17672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:18.275238 17672 hybrid_clock.cc:648] HybridClock initialized: now 1781836878275220 us; error 41 us; skew 500 ppm
Jun 19 02:41:18 dist-test-slave-zpbg krb5kdc[16841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836878, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:41:18.279264 17672 init.cc:377] Logged in from keytab as kudu/127.5.241.67@KRBTEST.COM (short username kudu)
I20260619 02:41:18.279832 17672 webserver.cc:460] Webserver started at http://127.5.241.67:33283/ using document root <none> and password file <none>
I20260619 02:41:18.280130 17672 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:18.280256 17672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:18.280434 17672 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 02:41:18 dist-test-slave-zpbg krb5kdc[16841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836878, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.67@KRBTEST.COM for HTTP/127.5.241.84@KRBTEST.COM
I20260619 02:41:18.301290 17672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ts-2/data/instance:
uuid: "36ce0981271d4ebcac27c16cb1382885"
format_stamp: "Formatted at 2026-06-19 02:41:18 on dist-test-slave-zpbg"
server_key: "03845f21483e480982cd004c0beb2c85"
server_key_iv: "6aff8e227e076d1796c90e3eef50aa3c"
server_key_version: "kuduclusterkey@0"
I20260619 02:41:18.301700 17672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ts-2/wal/instance:
uuid: "36ce0981271d4ebcac27c16cb1382885"
format_stamp: "Formatted at 2026-06-19 02:41:18 on dist-test-slave-zpbg"
server_key: "03845f21483e480982cd004c0beb2c85"
server_key_iv: "6aff8e227e076d1796c90e3eef50aa3c"
server_key_version: "kuduclusterkey@0"
I20260619 02:41:18.303310 17672 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:41:18.322889 17687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:18.323167 17672 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.001s sys 0.000s
I20260619 02:41:18.323303 17672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ts-2/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ts-2/wal
uuid: "36ce0981271d4ebcac27c16cb1382885"
format_stamp: "Formatted at 2026-06-19 02:41:18 on dist-test-slave-zpbg"
server_key: "03845f21483e480982cd004c0beb2c85"
server_key_iv: "6aff8e227e076d1796c90e3eef50aa3c"
server_key_version: "kuduclusterkey@0"
I20260619 02:41:18.323469 17672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:18.337529 17672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:18.339838 17672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:18.339985 17672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:18.340197 17672 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:41:18.340562 17672 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:41:18.340608 17672 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:18.340691 17672 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:41:18.341106 17672 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:18.348277 17672 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.67:39693
I20260619 02:41:18.348728 17672 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
I20260619 02:41:18.350418 17800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.67:39693 every 8 connection(s)
I20260619 02:41:18.352949 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 17672
I20260619 02:41:18.353062 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ts-2/wal/instance
I20260619 02:41:18.353242 6085 external_mini_cluster.cc:1428] Setting key 29ae750b62146223a8e72a6621c106af
Jun 19 02:41:18 dist-test-slave-zpbg krb5kdc[16841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836878, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.67@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:41:18.362295 17532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:18.350867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:46621 (local address 127.5.241.126:36063)
0619 02:41:18.350920 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:18.350922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:18.351051 (+ 129us) server_negotiation.cc:407] Connection header received
0619 02:41:18.352202 (+ 1151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:18.352204 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:18.352219 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:18.352305 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:18.353059 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.353779 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.355254 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.355562 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.355641 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:18.358345 (+ 2704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:18.358347 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:18.358347 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:18.358356 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:18.360278 (+ 1922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:18.361025 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:18.361027 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:18.361028 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:18.361105 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:18.361418 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:18.361420 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:18.361420 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:18.361592 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:18.361675 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:18.362073 (+ 398us) server_negotiation.cc:299] Negotiation successful
0619 02:41:18.362174 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 02:41:18.362443 17803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:18.350528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:36063 (local address 127.5.241.67:46621)
0619 02:41:18.350922 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:18.350966 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:18.352057 (+ 1091us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:18.352394 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:18.352399 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:18.352638 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:18.352942 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:18.352948 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.353900 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.353901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:18.355175 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:18.355178 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.355660 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.355661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:18.355819 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:18.356432 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:18.356441 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:18.358213 (+ 1772us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:18.360378 (+ 2165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:18.360385 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:18.360388 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:18.360688 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:18.361190 (+ 502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:18.361191 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:18.361192 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:18.361338 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:18.361670 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:18.361675 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:18.361917 (+ 242us) client_negotiation.cc:769] Sending connection context
0619 02:41:18.362032 (+ 115us) client_negotiation.cc:240] Negotiation successful
0619 02:41:18.362189 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":95,"threads_started":1}
I20260619 02:41:18.363718 17801 heartbeater.cc:347] Connected to a master server at 127.5.241.126:36063
I20260619 02:41:18.363853 17801 heartbeater.cc:464] Registering TS with master...
I20260619 02:41:18.364079 17801 heartbeater.cc:511] Master 127.5.241.126:36063 requested a full tablet report, sending...
I20260619 02:41:18.364511 17317 ts_manager.cc:194] Registered new tserver with Master: 36ce0981271d4ebcac27c16cb1382885 (127.5.241.67:39693)
I20260619 02:41:18.365054 17317 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.67@KRBTEST.COM'} at 127.5.241.67:46621
I20260619 02:41:18.365653 6085 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:41:18.383311 17532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:18.374851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36498 (local address 127.5.241.126:36063)
0619 02:41:18.374900 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:18.374902 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:18.375190 (+ 288us) server_negotiation.cc:407] Connection header received
0619 02:41:18.375312 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:18.375313 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:18.375325 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:18.375355 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:18.376245 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.377043 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.378175 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.378488 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.378532 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:18.380188 (+ 1656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:18.380190 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:18.380190 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:18.380198 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:18.381953 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:18.382419 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:18.382420 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:18.382420 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:18.382479 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:18.382675 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:18.382676 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:18.382677 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:18.382824 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:18.382903 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:18.383120 (+ 217us) server_negotiation.cc:299] Negotiation successful
0619 02:41:18.383198 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 02:41:18.385613 17317 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36498:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:41:18.387925 17317 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:41:18.407550 17818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:18.391401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:44921 (local address 127.0.0.1:35576)
0619 02:41:18.393295 (+ 1894us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:18.393304 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:18.393355 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:18.393711 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:18.393714 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:18.393716 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:41:18.393788 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:18.393790 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.402317 (+ 8527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.402318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:18.403799 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:18.403802 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.407184 (+ 3382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.407185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:18.407313 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:18.407330 (+ 17us) client_negotiation.cc:769] Sending connection context
0619 02:41:18.407398 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 02:41:18.407454 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1879,"thread_start_us":67,"threads_started":1}
I20260619 02:41:18.408001 17815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:18.391189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.67:39693 (local address 127.0.0.1:42506)
0619 02:41:18.392733 (+ 1544us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:18.392776 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:18.392838 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:18.393862 (+ 1024us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:18.393863 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:18.393864 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:41:18.393894 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:18.393895 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.396204 (+ 2309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.396205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:18.402191 (+ 5986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:18.402193 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.407778 (+ 5585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.407779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:18.407839 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:18.407843 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:41:18.407885 (+ 42us) client_negotiation.cc:240] Negotiation successful
0619 02:41:18.407929 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1514,"thread_start_us":82,"threads_started":1}
I20260619 02:41:18.408483 17814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:18.392132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.66:46269 (local address 127.0.0.1:41002)
0619 02:41:18.397764 (+ 5632us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:18.397777 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:18.397835 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:18.398886 (+ 1051us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:18.398887 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:18.398889 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:41:18.398932 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:18.398934 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.400588 (+ 1654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.400589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:18.401811 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:18.401814 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.408245 (+ 6431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.408246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:18.408310 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:18.408313 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:41:18.408380 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 02:41:18.408421 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5612,"thread_start_us":62,"threads_started":1}
I20260619 02:41:18.408529 17819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:18.392383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41002 (local address 127.5.241.66:46269)
0619 02:41:18.393474 (+ 1091us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:18.393479 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:18.397901 (+ 4422us) server_negotiation.cc:407] Connection header received
0619 02:41:18.397950 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:18.397953 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:18.397970 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:18.398045 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:41:18.399019 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.400485 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.404958 (+ 4473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.405754 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.405824 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:18.405890 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:18.408401 (+ 2511us) server_negotiation.cc:299] Negotiation successful
0619 02:41:18.408445 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1054,"thread_start_us":70,"threads_started":1}
I20260619 02:41:18.408926 17817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:18.391863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35576 (local address 127.5.241.65:44921)
0619 02:41:18.393242 (+ 1379us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:18.393247 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:18.393501 (+ 254us) server_negotiation.cc:407] Connection header received
0619 02:41:18.393538 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:18.393540 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:18.393556 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:18.393628 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:41:18.394553 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.398692 (+ 4139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.405928 (+ 7236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.406842 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.406914 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:18.408807 (+ 1893us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:18.408832 (+ 25us) server_negotiation.cc:299] Negotiation successful
0619 02:41:18.408869 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1334,"thread_start_us":48,"threads_started":1}
I20260619 02:41:18.409209 17816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:18.391677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42506 (local address 127.5.241.67:39693)
0619 02:41:18.392978 (+ 1301us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:18.392983 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:18.392993 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:41:18.393030 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:18.393032 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:18.393048 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:18.393112 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:41:18.393966 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.396101 (+ 2135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.403926 (+ 7825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.404752 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.404828 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:18.404907 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:18.409068 (+ 4161us) server_negotiation.cc:299] Negotiation successful
0619 02:41:18.409119 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1264,"thread_start_us":81,"threads_started":1}
I20260619 02:41:18.409998 17728 tablet_service.cc:1467] Processing CreateTablet for tablet ab3fd9774f9e4d6aa9c2809c1639f5d0 (DEFAULT_TABLE table=test-table [id=eff158c1af4d44c8b03822f122a80005]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:41:18.410041 17601 tablet_service.cc:1467] Processing CreateTablet for tablet ab3fd9774f9e4d6aa9c2809c1639f5d0 (DEFAULT_TABLE table=test-table [id=eff158c1af4d44c8b03822f122a80005]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:41:18.410364 17601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab3fd9774f9e4d6aa9c2809c1639f5d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:41:18.410364 17728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab3fd9774f9e4d6aa9c2809c1639f5d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:41:18.415419 17463 tablet_service.cc:1467] Processing CreateTablet for tablet ab3fd9774f9e4d6aa9c2809c1639f5d0 (DEFAULT_TABLE table=test-table [id=eff158c1af4d44c8b03822f122a80005]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:41:18.415751 17463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab3fd9774f9e4d6aa9c2809c1639f5d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:41:18.416790 17821 tablet_bootstrap.cc:492] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 36ce0981271d4ebcac27c16cb1382885: Bootstrap starting.
I20260619 02:41:18.417829 17821 tablet_bootstrap.cc:654] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 36ce0981271d4ebcac27c16cb1382885: Neither blocks nor log segments found. Creating new log.
I20260619 02:41:18.418638 17821 log.cc:824] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 36ce0981271d4ebcac27c16cb1382885: Log is configured to *not* fsync() on all Append() calls
I20260619 02:41:18.419860 17821 tablet_bootstrap.cc:492] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 36ce0981271d4ebcac27c16cb1382885: No bootstrap required, opened a new log
I20260619 02:41:18.419935 17821 ts_tablet_manager.cc:1397] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 36ce0981271d4ebcac27c16cb1382885: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260619 02:41:18.420328 17820 tablet_bootstrap.cc:492] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 47bf02c7985044b29cfd995c0552099b: Bootstrap starting.
I20260619 02:41:18.421351 17820 tablet_bootstrap.cc:654] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 47bf02c7985044b29cfd995c0552099b: Neither blocks nor log segments found. Creating new log.
I20260619 02:41:18.421636 17820 log.cc:824] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 47bf02c7985044b29cfd995c0552099b: Log is configured to *not* fsync() on all Append() calls
I20260619 02:41:18.421936 17823 tablet_bootstrap.cc:492] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 9bbd3bc6d19647e1b487a2b123cabe6a: Bootstrap starting.
I20260619 02:41:18.422104 17821 raft_consensus.cc:348] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 36ce0981271d4ebcac27c16cb1382885 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36ce0981271d4ebcac27c16cb1382885" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 39693 } } peers { permanent_uuid: "9bbd3bc6d19647e1b487a2b123cabe6a" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 44921 } } peers { permanent_uuid: "47bf02c7985044b29cfd995c0552099b" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 46269 } }
I20260619 02:41:18.422279 17821 raft_consensus.cc:374] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 36ce0981271d4ebcac27c16cb1382885 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:41:18.422317 17821 raft_consensus.cc:729] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 36ce0981271d4ebcac27c16cb1382885 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36ce0981271d4ebcac27c16cb1382885, State: Initialized, Role: FOLLOWER
I20260619 02:41:18.422447 17821 consensus_queue.cc:260] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 36ce0981271d4ebcac27c16cb1382885 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36ce0981271d4ebcac27c16cb1382885" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 39693 } } peers { permanent_uuid: "9bbd3bc6d19647e1b487a2b123cabe6a" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 44921 } } peers { permanent_uuid: "47bf02c7985044b29cfd995c0552099b" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 46269 } }
I20260619 02:41:18.422580 17823 tablet_bootstrap.cc:654] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 9bbd3bc6d19647e1b487a2b123cabe6a: Neither blocks nor log segments found. Creating new log.
I20260619 02:41:18.422711 17821 ts_tablet_manager.cc:1428] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 36ce0981271d4ebcac27c16cb1382885: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 02:41:18.422883 17823 log.cc:824] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 9bbd3bc6d19647e1b487a2b123cabe6a: Log is configured to *not* fsync() on all Append() calls
I20260619 02:41:18.423170 17801 heartbeater.cc:503] Master 127.5.241.126:36063 was elected leader, sending a full tablet report...
I20260619 02:41:18.423617 17820 tablet_bootstrap.cc:492] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 47bf02c7985044b29cfd995c0552099b: No bootstrap required, opened a new log
I20260619 02:41:18.423703 17820 ts_tablet_manager.cc:1397] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 47bf02c7985044b29cfd995c0552099b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 02:41:18.425926 17820 raft_consensus.cc:348] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 47bf02c7985044b29cfd995c0552099b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36ce0981271d4ebcac27c16cb1382885" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 39693 } } peers { permanent_uuid: "9bbd3bc6d19647e1b487a2b123cabe6a" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 44921 } } peers { permanent_uuid: "47bf02c7985044b29cfd995c0552099b" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 46269 } }
I20260619 02:41:18.426079 17820 raft_consensus.cc:374] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 47bf02c7985044b29cfd995c0552099b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:41:18.426141 17820 raft_consensus.cc:729] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 47bf02c7985044b29cfd995c0552099b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47bf02c7985044b29cfd995c0552099b, State: Initialized, Role: FOLLOWER
I20260619 02:41:18.426288 17820 consensus_queue.cc:260] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 47bf02c7985044b29cfd995c0552099b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36ce0981271d4ebcac27c16cb1382885" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 39693 } } peers { permanent_uuid: "9bbd3bc6d19647e1b487a2b123cabe6a" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 44921 } } peers { permanent_uuid: "47bf02c7985044b29cfd995c0552099b" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 46269 } }
I20260619 02:41:18.426561 17820 ts_tablet_manager.cc:1428] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 47bf02c7985044b29cfd995c0552099b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 02:41:18.426601 17667 heartbeater.cc:503] Master 127.5.241.126:36063 was elected leader, sending a full tablet report...
I20260619 02:41:18.427217 17823 tablet_bootstrap.cc:492] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 9bbd3bc6d19647e1b487a2b123cabe6a: No bootstrap required, opened a new log
I20260619 02:41:18.427680 17823 ts_tablet_manager.cc:1397] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 9bbd3bc6d19647e1b487a2b123cabe6a: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260619 02:41:18.430370 17823 raft_consensus.cc:348] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 9bbd3bc6d19647e1b487a2b123cabe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36ce0981271d4ebcac27c16cb1382885" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 39693 } } peers { permanent_uuid: "9bbd3bc6d19647e1b487a2b123cabe6a" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 44921 } } peers { permanent_uuid: "47bf02c7985044b29cfd995c0552099b" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 46269 } }
I20260619 02:41:18.430588 17823 raft_consensus.cc:374] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 9bbd3bc6d19647e1b487a2b123cabe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:41:18.430662 17823 raft_consensus.cc:729] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 9bbd3bc6d19647e1b487a2b123cabe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bbd3bc6d19647e1b487a2b123cabe6a, State: Initialized, Role: FOLLOWER
I20260619 02:41:18.430857 17823 consensus_queue.cc:260] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 9bbd3bc6d19647e1b487a2b123cabe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36ce0981271d4ebcac27c16cb1382885" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 39693 } } peers { permanent_uuid: "9bbd3bc6d19647e1b487a2b123cabe6a" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 44921 } } peers { permanent_uuid: "47bf02c7985044b29cfd995c0552099b" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 46269 } }
I20260619 02:41:18.431191 17823 ts_tablet_manager.cc:1428] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 9bbd3bc6d19647e1b487a2b123cabe6a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 02:41:18.431353 17529 heartbeater.cc:503] Master 127.5.241.126:36063 was elected leader, sending a full tablet report...
I20260619 02:41:18.497378 17825 raft_consensus.cc:482] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 36ce0981271d4ebcac27c16cb1382885 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:41:18.497534 17825 raft_consensus.cc:504] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 36ce0981271d4ebcac27c16cb1382885 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36ce0981271d4ebcac27c16cb1382885" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 39693 } } peers { permanent_uuid: "9bbd3bc6d19647e1b487a2b123cabe6a" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 44921 } } peers { permanent_uuid: "47bf02c7985044b29cfd995c0552099b" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 46269 } }
I20260619 02:41:18.497967 17825 leader_election.cc:288] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 36ce0981271d4ebcac27c16cb1382885 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9bbd3bc6d19647e1b487a2b123cabe6a (127.5.241.65:44921), 47bf02c7985044b29cfd995c0552099b (127.5.241.66:46269)
I20260619 02:41:18.502652 17803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:18.498311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.66:46269 (local address 127.5.241.67:36805)
0619 02:41:18.498367 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:18.498384 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:18.498450 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:18.499114 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:18.499115 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:18.499117 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:41:18.499209 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:18.499212 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.500380 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.500381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:18.501667 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:18.501670 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.502401 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.502402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:18.502484 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:18.502488 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:41:18.502536 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 02:41:18.502577 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260619 02:41:18.503224 17819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:18.498548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:36805 (local address 127.5.241.66:46269)
0619 02:41:18.498591 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:18.498594 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:18.498604 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:41:18.498635 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:18.498636 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:18.498650 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:18.498712 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:41:18.499352 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.500096 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.501767 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.502272 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.502342 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:18.502352 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:18.503114 (+ 762us) server_negotiation.cc:299] Negotiation successful
0619 02:41:18.503160 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 02:41:18.503820 17605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab3fd9774f9e4d6aa9c2809c1639f5d0" candidate_uuid: "36ce0981271d4ebcac27c16cb1382885" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47bf02c7985044b29cfd995c0552099b" is_pre_election: true
I20260619 02:41:18.503983 17605 raft_consensus.cc:2454] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 47bf02c7985044b29cfd995c0552099b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36ce0981271d4ebcac27c16cb1382885 in term 0.
I20260619 02:41:18.504269 17690 leader_election.cc:302] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 36ce0981271d4ebcac27c16cb1382885 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36ce0981271d4ebcac27c16cb1382885, 47bf02c7985044b29cfd995c0552099b; no voters:
I20260619 02:41:18.504535 17825 raft_consensus.cc:2781] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 36ce0981271d4ebcac27c16cb1382885 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:41:18.504599 17825 raft_consensus.cc:482] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 36ce0981271d4ebcac27c16cb1382885 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:41:18.504624 17825 raft_consensus.cc:3037] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 36ce0981271d4ebcac27c16cb1382885 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:41:18.505514 17825 raft_consensus.cc:504] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 36ce0981271d4ebcac27c16cb1382885 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36ce0981271d4ebcac27c16cb1382885" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 39693 } } peers { permanent_uuid: "9bbd3bc6d19647e1b487a2b123cabe6a" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 44921 } } peers { permanent_uuid: "47bf02c7985044b29cfd995c0552099b" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 46269 } }
I20260619 02:41:18.505643 17825 leader_election.cc:288] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 36ce0981271d4ebcac27c16cb1382885 [CANDIDATE]: Term 1 election: Requested vote from peers 9bbd3bc6d19647e1b487a2b123cabe6a (127.5.241.65:44921), 47bf02c7985044b29cfd995c0552099b (127.5.241.66:46269)
I20260619 02:41:18.506157 17605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab3fd9774f9e4d6aa9c2809c1639f5d0" candidate_uuid: "36ce0981271d4ebcac27c16cb1382885" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47bf02c7985044b29cfd995c0552099b"
I20260619 02:41:18.506260 17605 raft_consensus.cc:3037] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 47bf02c7985044b29cfd995c0552099b [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:41:18.507148 17605 raft_consensus.cc:2454] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 47bf02c7985044b29cfd995c0552099b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36ce0981271d4ebcac27c16cb1382885 in term 1.
I20260619 02:41:18.507833 17690 leader_election.cc:302] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 36ce0981271d4ebcac27c16cb1382885 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36ce0981271d4ebcac27c16cb1382885, 47bf02c7985044b29cfd995c0552099b; no voters:
I20260619 02:41:18.507985 17825 raft_consensus.cc:2781] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 36ce0981271d4ebcac27c16cb1382885 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:41:18.508121 17825 raft_consensus.cc:686] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 36ce0981271d4ebcac27c16cb1382885 [term 1 LEADER]: Becoming Leader. State: Replica: 36ce0981271d4ebcac27c16cb1382885, State: Running, Role: LEADER
I20260619 02:41:18.508208 17825 consensus_queue.cc:237] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 36ce0981271d4ebcac27c16cb1382885 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36ce0981271d4ebcac27c16cb1382885" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 39693 } } peers { permanent_uuid: "9bbd3bc6d19647e1b487a2b123cabe6a" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 44921 } } peers { permanent_uuid: "47bf02c7985044b29cfd995c0552099b" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 46269 } }
I20260619 02:41:18.509027 17317 catalog_manager.cc:5495] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 36ce0981271d4ebcac27c16cb1382885 reported cstate change: term changed from 0 to 1, leader changed from <none> to 36ce0981271d4ebcac27c16cb1382885 (127.5.241.67). New cstate: current_term: 1 leader_uuid: "36ce0981271d4ebcac27c16cb1382885" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36ce0981271d4ebcac27c16cb1382885" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 39693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9bbd3bc6d19647e1b487a2b123cabe6a" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 44921 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47bf02c7985044b29cfd995c0552099b" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 46269 } health_report { overall_health: UNKNOWN } } }
I20260619 02:41:18.513592 17803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:18.500209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:44921 (local address 127.5.241.67:52215)
0619 02:41:18.502726 (+ 2517us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:18.502737 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:18.510289 (+ 7552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:18.510484 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:18.510485 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:18.510487 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:41:18.510527 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:18.510529 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.511457 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.511458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:18.512567 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:18.512570 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.513360 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.513361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:18.513420 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:18.513424 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:41:18.513475 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 02:41:18.513514 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2506,"thread_start_us":121,"threads_started":1}
I20260619 02:41:18.513914 17817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:18.498976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:52215 (local address 127.5.241.65:44921)
0619 02:41:18.499037 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:18.499039 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:18.502815 (+ 3776us) server_negotiation.cc:407] Connection header received
0619 02:41:18.510381 (+ 7566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:18.510382 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:18.510394 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:18.510439 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:41:18.510596 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.511354 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.512668 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.513224 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.513312 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:18.513326 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:18.513805 (+ 479us) server_negotiation.cc:299] Negotiation successful
0619 02:41:18.513852 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 02:41:18.514446 17483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab3fd9774f9e4d6aa9c2809c1639f5d0" candidate_uuid: "36ce0981271d4ebcac27c16cb1382885" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bbd3bc6d19647e1b487a2b123cabe6a" is_pre_election: true
I20260619 02:41:18.514607 17483 raft_consensus.cc:2454] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 9bbd3bc6d19647e1b487a2b123cabe6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36ce0981271d4ebcac27c16cb1382885 in term 0.
I20260619 02:41:18.514751 17482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab3fd9774f9e4d6aa9c2809c1639f5d0" candidate_uuid: "36ce0981271d4ebcac27c16cb1382885" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bbd3bc6d19647e1b487a2b123cabe6a"
I20260619 02:41:18.514839 17482 raft_consensus.cc:3037] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 9bbd3bc6d19647e1b487a2b123cabe6a [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:41:18.515770 17482 raft_consensus.cc:2454] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 9bbd3bc6d19647e1b487a2b123cabe6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36ce0981271d4ebcac27c16cb1382885 in term 1.
Jun 19 02:41:18 dist-test-slave-zpbg krb5kdc[16841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836877, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for HTTP/127.5.241.84@KRBTEST.COM
I20260619 02:41:18.554724 17816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:18.551454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42510 (local address 127.5.241.67:39693)
0619 02:41:18.551509 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:18.551511 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:18.551523 (+ 12us) server_negotiation.cc:407] Connection header received
0619 02:41:18.551584 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:18.551585 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:18.551609 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:18.551696 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:41:18.551902 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.552604 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.553630 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.553919 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.553986 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:18.554199 (+ 213us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:41:18.554355 (+ 156us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:41:18.554433 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:18.554604 (+ 171us) server_negotiation.cc:299] Negotiation successful
0619 02:41:18.554623 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 02:41:18.565613 17605 raft_consensus.cc:1261] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 47bf02c7985044b29cfd995c0552099b [term 1 FOLLOWER]: Refusing update from remote peer 36ce0981271d4ebcac27c16cb1382885: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:41:18.566377 17482 raft_consensus.cc:1261] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 9bbd3bc6d19647e1b487a2b123cabe6a [term 1 FOLLOWER]: Refusing update from remote peer 36ce0981271d4ebcac27c16cb1382885: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:41:18.566560 17830 consensus_queue.cc:1035] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 36ce0981271d4ebcac27c16cb1382885 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47bf02c7985044b29cfd995c0552099b" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 46269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:41:18.567134 17830 consensus_queue.cc:1035] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 36ce0981271d4ebcac27c16cb1382885 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bbd3bc6d19647e1b487a2b123cabe6a" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 44921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:41:18.569811 17833 mvcc.cc:205] Tried to move back new op lower bound from 7298403854593912832 to 7298403854370295808. Current Snapshot: MvccSnapshot[applied={T|T < 7298403854593912832}]
I20260619 02:41:18.571568 17834 mvcc.cc:205] Tried to move back new op lower bound from 7298403854593912832 to 7298403854370295808. Current Snapshot: MvccSnapshot[applied={T|T < 7298403854593912832}]
I20260619 02:41:18.582409 17836 mvcc.cc:205] Tried to move back new op lower bound from 7298403854593912832 to 7298403854370295808. Current Snapshot: MvccSnapshot[applied={T|T < 7298403854593912832}]
I20260619 02:41:18.582665 17316 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36498:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:41:18.582796 17316 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36498:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:41:18.583570 17316 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 798799cac97345efb7c01712070d2d70: Sending DeleteTablet for 3 replicas of tablet ab3fd9774f9e4d6aa9c2809c1639f5d0
I20260619 02:41:18.583950 17463 tablet_service.cc:1514] Processing DeleteTablet for tablet ab3fd9774f9e4d6aa9c2809c1639f5d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:41:18 UTC) from {username='kudu', principal='kudu/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:35576
I20260619 02:41:18.584050 17601 tablet_service.cc:1514] Processing DeleteTablet for tablet ab3fd9774f9e4d6aa9c2809c1639f5d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:41:18 UTC) from {username='kudu', principal='kudu/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:41002
I20260619 02:41:18.584237 17728 tablet_service.cc:1514] Processing DeleteTablet for tablet ab3fd9774f9e4d6aa9c2809c1639f5d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:41:18 UTC) from {username='kudu', principal='kudu/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:42506
I20260619 02:41:18.585232 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 17398
I20260619 02:41:18.592450 17843 tablet_replica.cc:330] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 47bf02c7985044b29cfd995c0552099b: stopping tablet replica
I20260619 02:41:18.592782 17844 tablet_replica.cc:330] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 36ce0981271d4ebcac27c16cb1382885: stopping tablet replica
I20260619 02:41:18.593483 17843 raft_consensus.cc:2229] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 47bf02c7985044b29cfd995c0552099b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:41:18.593614 17844 raft_consensus.cc:2229] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 36ce0981271d4ebcac27c16cb1382885 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:41:18.593822 17844 raft_consensus.cc:2258] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 36ce0981271d4ebcac27c16cb1382885 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:41:18.593984 17843 raft_consensus.cc:2258] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 47bf02c7985044b29cfd995c0552099b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:41:18.595907 17844 ts_tablet_manager.cc:1905] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 36ce0981271d4ebcac27c16cb1382885: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:41:18.598430 17843 ts_tablet_manager.cc:1905] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 47bf02c7985044b29cfd995c0552099b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:41:18.601831 17844 ts_tablet_manager.cc:1918] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 36ce0981271d4ebcac27c16cb1382885: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:41:18.602763 17843 ts_tablet_manager.cc:1918] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 47bf02c7985044b29cfd995c0552099b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:41:18.606853 17844 log.cc:1196] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 36ce0981271d4ebcac27c16cb1382885: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ts-2/wal/wals/ab3fd9774f9e4d6aa9c2809c1639f5d0
I20260619 02:41:18.607015 17843 log.cc:1196] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 47bf02c7985044b29cfd995c0552099b: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836772852275-6085-0/minicluster-data/ts-1/wal/wals/ab3fd9774f9e4d6aa9c2809c1639f5d0
I20260619 02:41:18.615131 17844 ts_tablet_manager.cc:1939] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 36ce0981271d4ebcac27c16cb1382885: Deleting consensus metadata
I20260619 02:41:18.615535 17843 ts_tablet_manager.cc:1939] T ab3fd9774f9e4d6aa9c2809c1639f5d0 P 47bf02c7985044b29cfd995c0552099b: Deleting consensus metadata
I20260619 02:41:18.616129 17303 catalog_manager.cc:4841] TS 47bf02c7985044b29cfd995c0552099b (127.5.241.66:46269): tablet ab3fd9774f9e4d6aa9c2809c1639f5d0 (table test-table [id=eff158c1af4d44c8b03822f122a80005]) successfully deleted
I20260619 02:41:18.616741 17304 catalog_manager.cc:4841] TS 36ce0981271d4ebcac27c16cb1382885 (127.5.241.67:39693): tablet ab3fd9774f9e4d6aa9c2809c1639f5d0 (table test-table [id=eff158c1af4d44c8b03822f122a80005]) successfully deleted
W20260619 02:41:18.617043 17304 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.241.65:44921 (error 108)
I20260619 02:41:18.618903 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 17535
I20260619 02:41:18.625433 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 17672
I20260619 02:41:18.632210 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 17285
I20260619 02:41:18.638479 6085 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260619 02:41:18.983019 6085 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260619 02:41:18.983139 6085 mini_ranger.cc:66] Stopping Ranger...
I20260619 02:41:19.327678 6085 mini_ranger.cc:68] Stopped Ranger
2026-06-19 02:41:19.327 UTC [16871] LOG: received smart shutdown request
2026-06-19 02:41:19.331 UTC [16871] LOG: background worker "logical replication launcher" (PID 16880) exited with exit code 1
2026-06-19 02:41:19.331 UTC [16875] LOG: shutting down
2026-06-19 02:41:19.512 UTC [16871] LOG: database system is shut down
2026-06-19T02:41:19Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (46765 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:41:19 dist-test-slave-zpbg krb5kdc[17854](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:41:19 dist-test-slave-zpbg krb5kdc[17854](info): set up 2 sockets
Jun 19 02:41:19 dist-test-slave-zpbg krb5kdc[17854](info): commencing operation
krb5kdc: starting...
W20260619 02:41:20.655063 6085 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Jun 19 02:41:20 dist-test-slave-zpbg krb5kdc[17854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:41:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:41:20Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.5.241.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.5.241.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.5.241.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/rangeradmin_127.5.241.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.5.241.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/rangeradmin_127.5.241.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.5.241.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.5.241.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.5.241.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/rangerlookup_127.5.241.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.5.241.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/rangerlookup_127.5.241.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.241.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.5.241.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.241.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/HTTP_127.5.241.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.241.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/HTTP_127.5.241.84@KRBTEST.COM.keytab.
I20260619 02:41:20.825831 6085 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskflgrG6/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/postgres -l logfile start
2026-06-19 02:41:23.434 UTC [17884] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-19 02:41:23.434 UTC [17884] LOG: listening on IPv4 address "127.5.241.84", port 51671
2026-06-19 02:41:23.437 UTC [17884] LOG: listening on Unix socket "/tmp/.s.PGSQL.51671"
2026-06-19 02:41:23.459 UTC [17887] LOG: database system was shut down at 2026-06-19 02:41:21 UTC
2026-06-19 02:41:23.462 UTC [17884] LOG: database system is ready to accept connections
I20260619 02:41:24.422593 6085 mini_postgres.cc:96] Postgres bound to 51671
2026-06-19 02:41:24.428 UTC [17895] FATAL: database "slave" does not exist
127.5.241.84:51671 - accepting connections
I20260619 02:41:24.429214 6085 mini_ranger.cc:161] Starting Ranger...
I20260619 02:41:24.443715 6085 mini_ranger.cc:84] Created miniranger Postgres user
I20260619 02:41:24.921346 6085 mini_ranger.cc:87] Created ranger Postgres database
I20260619 02:41:24.921433 6085 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ranger-admin
2026-06-19 02:41:25,165 [I] DB FLAVOR :POSTGRES
2026-06-19 02:41:25,165 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 02:41:25,166 [I] Checking connection..
2026-06-19 02:41:25,166 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskflgrG6/build/release/bin/postgresql.jar:/tmp/dist-test-taskflgrG6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.241.84:51671/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-19 02:41:25,406 [I] Checking connection passed.
2026-06-19 02:41:25,406 [I] --------- Verifying version history table ---------
2026-06-19 02:41:25,406 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskflgrG6/build/release/bin/postgresql.jar:/tmp/dist-test-taskflgrG6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.241.84:51671/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 02:41:25,666 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 02:41:25,666 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskflgrG6/build/release/bin/postgresql.jar:/tmp/dist-test-taskflgrG6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.241.84:51671/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 02:41:25,930 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 02:41:25,930 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-19 02:41:25,930 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskflgrG6/build/release/bin/postgresql.jar:/tmp/dist-test-taskflgrG6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.241.84:51671/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskflgrG6/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-19 02:41:26.159 UTC [17959] WARNING: there is no transaction in progress
2026-06-19 02:41:26,166 [I] create_dbversion_catalog.sql file imported successfully
2026-06-19 02:41:26,166 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskflgrG6/build/release/bin/postgresql.jar:/tmp/dist-test-taskflgrG6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.241.84:51671/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 02:41:26,423 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-19 02:41:26,423 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 02:41:26,423 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskflgrG6/build/release/bin/postgresql.jar:/tmp/dist-test-taskflgrG6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.241.84:51671/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-19 02:41:26,679 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskflgrG6/build/release/bin/postgresql.jar:/tmp/dist-test-taskflgrG6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.241.84:51671/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-19 02:41:26,932 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskflgrG6/build/release/bin/postgresql.jar:/tmp/dist-test-taskflgrG6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.241.84:51671/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-zpbg.c.gcp-upstream.internal','N') ;"
2026-06-19 02:41:27,150 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskflgrG6/build/release/bin/postgresql.jar:/tmp/dist-test-taskflgrG6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.241.84:51671/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 02:41:27,411 [I] Table x_portal_user does not exist in database ranger
2026-06-19 02:41:27,411 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskflgrG6/build/release/bin/postgresql.jar:/tmp/dist-test-taskflgrG6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.241.84:51671/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 02:41:27,642 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-19 02:41:27,642 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskflgrG6/build/release/bin/postgresql.jar:/tmp/dist-test-taskflgrG6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.241.84:51671/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 02:41:27,897 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskflgrG6/build/release/bin/postgresql.jar:/tmp/dist-test-taskflgrG6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.241.84:51671/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 02:41:28,123 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-19 02:41:28,123 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskflgrG6/build/release/bin/postgresql.jar:/tmp/dist-test-taskflgrG6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.241.84:51671/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskflgrG6/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-19 02:41:29.127 UTC [18067] WARNING: there is no transaction in progress
2026-06-19 02:41:29.146 UTC [18067] WARNING: there is no transaction in progress
2026-06-19 02:41:29.163 UTC [18067] WARNING: there is no transaction in progress
2026-06-19 02:41:29.182 UTC [18067] WARNING: there is no transaction in progress
2026-06-19 02:41:29.204 UTC [18067] WARNING: there is no transaction in progress
2026-06-19 02:41:29.225 UTC [18067] WARNING: there is no transaction in progress
2026-06-19 02:41:29.283 UTC [18067] WARNING: there is no transaction in progress
2026-06-19 02:41:29.290 UTC [18067] WARNING: there is no transaction in progress
2026-06-19 02:41:29.306 UTC [18067] WARNING: there is no transaction in progress
2026-06-19 02:41:29.317 UTC [18067] WARNING: there is no transaction in progress
2026-06-19 02:41:29.327 UTC [18067] WARNING: there is no transaction in progress
2026-06-19 02:41:29.341 UTC [18067] WARNING: there is no transaction in progress
2026-06-19 02:41:29.351 UTC [18067] WARNING: there is no transaction in progress
2026-06-19 02:41:29.358 UTC [18067] WARNING: there is no transaction in progress
2026-06-19 02:41:29.365 UTC [18067] WARNING: there is no transaction in progress
2026-06-19 02:41:29.372 UTC [18067] WARNING: there is no transaction in progress
2026-06-19 02:41:30,044 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-19 02:41:30,044 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskflgrG6/build/release/bin/postgresql.jar:/tmp/dist-test-taskflgrG6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.241.84:51671/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-19 02:41:30,288 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-19 02:41:30,289 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskflgrG6/build/release/bin/postgresql.jar:/tmp/dist-test-taskflgrG6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.241.84:51671/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 02:41:30,520 [I] Table x_portal_user already exists in database 'ranger'
2026-06-19 02:41:30,521 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskflgrG6/build/release/bin/postgresql.jar:/tmp/dist-test-taskflgrG6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.241.84:51671/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 02:41:30,767 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-19 02:41:30,767 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskflgrG6/build/release/bin/postgresql.jar:/tmp/dist-test-taskflgrG6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.241.84:51671/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 02:41:31,014 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskflgrG6/build/release/bin/postgresql.jar:/tmp/dist-test-taskflgrG6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.241.84:51671/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 02:41:31,249 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskflgrG6/build/release/bin/postgresql.jar:/tmp/dist-test-taskflgrG6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.241.84:51671/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-zpbg.c.gcp-upstream.internal';"
2026-06-19 02:41:31,480 [I] CORE_DB_SCHEMA import status has been updated
2026-06-19 02:41:31,481 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskflgrG6/build/release/bin/postgresql.jar:/tmp/dist-test-taskflgrG6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.241.84:51671/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-19 02:41:31,738 [I] DB_PATCHES have already been applied
I20260619 02:41:31.743446 6085 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ranger-admin:/tmp/dist-test-taskflgrG6/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskflgrG6/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskflgrG6/build/release/bin/postgresql.jar:/tmp/dist-test-taskflgrG6/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260619 02:41:31.743510 6085 mini_ranger.cc:193] Using host: 127.5.241.84
I20260619 02:41:31.746876 6085 mini_ranger.cc:227] Ranger admin URL: http://127.5.241.84:44037
Jun 19, 2026 2:41:32 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 2:41:32 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 2:41:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ranger-admin/ews/webapp
Jun 19, 2026 2:41:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 19, 2026 2:41:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 19, 2026 2:41:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ranger-admin/ews/webapp].
Jun 19, 2026 2:41:32 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260619 02:41:32.848356 6085 mini_ranger.cc:160] Time spent starting Ranger: real 8.419s user 0.000s sys 0.009s
Jun 19, 2026 2:41:53 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 19, 2026 2:41:53 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Jun 19, 2026 2:41:53 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 19, 2026 2:41:53 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 19, 2026 2:41:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 19, 2026 2:41:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 19, 2026 2:41:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 19, 2026 2:41:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 19, 2026 2:41:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 19, 2026 2:41:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 19, 2026 2:41:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 19, 2026 2:41:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 19, 2026 2:41:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 19, 2026 2:41:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 19, 2026 2:41:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 19, 2026 2:41:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 19, 2026 2:41:53 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 19, 2026 2:41:54 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260619 02:41:55.086977 6085 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.5.241.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.5.241.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.5.241.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/rangerkms_127.5.241.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.5.241.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/rangerkms_127.5.241.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.241.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.5.241.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.241.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/HTTP_127.5.241.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.241.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/HTTP_127.5.241.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 19 02:41:55 dist-test-slave-zpbg krb5kdc[17854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836915, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260619 02:41:55.166733 6085 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260619 02:41:55.175807 6085 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260619 02:41:56.056526 6085 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260619 02:41:56.056610 6085 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ranger-kms
I20260619 02:41:56.056632 6085 mini_ranger_kms.cc:227] Using postgres at 127.5.241.84:51671
2026-06-19 02:41:56,215 [I] DB FLAVOR :POSTGRES
2026-06-19 02:41:56,215 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 02:41:56,215 [I] Checking connection
2026-06-19 02:41:56,216 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskflgrG6/build/release/bin/postgresql.jar:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.241.84:51671/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-19 02:41:56,454 [I] connection success
2026-06-19 02:41:56,455 [I] --------- Verifying Ranger DB tables ---------
2026-06-19 02:41:56,455 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-19 02:41:56,455 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskflgrG6/build/release/bin/postgresql.jar:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.241.84:51671/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-19 02:41:56,718 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-19 02:41:56,718 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 02:41:56,718 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-19 02:41:56,718 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskflgrG6/build/release/bin/postgresql.jar:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.241.84:51671/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-19 02:41:56,980 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260619 02:41:57.461977 6085 mini_ranger_kms.cc:310] Created kms service
I20260619 02:41:57.681746 6085 mini_ranger_kms.cc:326] Created kudu user
I20260619 02:41:57.732029 6085 mini_ranger_kms.cc:343] Created rangerkms user
Jun 19 02:41:57 dist-test-slave-zpbg krb5kdc[17854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836915, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.5.241.84@KRBTEST.COM
I20260619 02:41:58.156152 6085 mini_ranger_kms.cc:384] Added ranger policy
I20260619 02:41:58.156494 6085 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ranger-kms:/tmp/dist-test-taskflgrG6/build/release/bin/postgresql.jar:/tmp/dist-test-taskflgrG6/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskflgrG6/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260619 02:41:58.156550 6085 mini_ranger_kms.cc:240] Using host: 127.5.241.84
I20260619 02:41:58.160621 6085 mini_ranger_kms.cc:276] Ranger KMS PID: 18247
I20260619 02:41:58.160729 6085 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.5.241.84:34815
Jun 19, 2026 2:41:59 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 2:41:59 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 2:41:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 19, 2026 2:41:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 19, 2026 2:41:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ranger-kms/ews/webapp].
Jun 19, 2026 2:41:59 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260619 02:42:00.402509 6085 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.236s user 0.012s sys 0.130s
I20260619 02:42:00.402622 6085 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260619 02:42:00.402673 6085 mini_ranger_kms.cc:401] 127.5.241.84:34815/kms/v1/keys
Jun 19 02:42:00 dist-test-slave-zpbg krb5kdc[17854](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781836920, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.241.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 19 02:42:00 dist-test-slave-zpbg krb5kdc[17854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836920, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.241.84@KRBTEST.COM for HTTP/127.5.241.84@KRBTEST.COM
Jun 19 02:42:01 dist-test-slave-zpbg krb5kdc[17854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836920, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.241.84@KRBTEST.COM for HTTP/127.5.241.84@KRBTEST.COM
Jun 19 02:42:03 dist-test-slave-zpbg krb5kdc[17854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836923, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.5.241.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
I20260619 02:42:03.490137 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:38663
--webserver_interface=127.5.241.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:44677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:38663
--ranger_config_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.241.84:34815/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:42:03.569273 18298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:42:03.569512 18298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:42:03.569546 18298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:42:03.570703 18298 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:42:03.570776 18298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:42:03.570828 18298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:42:03.570858 18298 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:42:03.570887 18298 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:42:03.572419 18298 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:44677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/master-0/wal
--ranger_kms_url=127.5.241.84:34815/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:38663
--ranger_config_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:38663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.241.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18298
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:42:03.572691 18298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:42:03.572921 18298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:42:03.577816 18304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:03.577819 18303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:03.577873 18306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:42:03.577948 18298 server_base.cc:1034] running on GCE node
I20260619 02:42:03.578163 18298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:42:03.578454 18298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:42:03.579595 18298 hybrid_clock.cc:648] HybridClock initialized: now 1781836923579578 us; error 39 us; skew 500 ppm
Jun 19 02:42:03 dist-test-slave-zpbg krb5kdc[17854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836923, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:42:03.586287 18298 init.cc:377] Logged in from keytab as kudu/127.5.241.126@KRBTEST.COM (short username kudu)
I20260619 02:42:03.586721 18298 webserver.cc:460] Webserver started at http://127.5.241.126:44925/ using document root <none> and password file <none>
I20260619 02:42:03.587003 18298 fs_manager.cc:359] Metadata directory not provided
I20260619 02:42:03.587047 18298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:42:03.587186 18298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 02:42:03 dist-test-slave-zpbg krb5kdc[17854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836923, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for HTTP/127.5.241.84@KRBTEST.COM
I20260619 02:42:03.641590 18298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/master-0/data/instance:
uuid: "bc6a5cfb7f254f74a8a1e7c431318d95"
format_stamp: "Formatted at 2026-06-19 02:42:03 on dist-test-slave-zpbg"
server_key: "f0978ec003c3a251b3d9f2aaa62f3a49"
server_key_iv: "c75b468f6b9fe3b840b5b0d581c65eba"
server_key_version: "kuduclusterkey@0"
I20260619 02:42:03.641990 18298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/master-0/wal/instance:
uuid: "bc6a5cfb7f254f74a8a1e7c431318d95"
format_stamp: "Formatted at 2026-06-19 02:42:03 on dist-test-slave-zpbg"
server_key: "f0978ec003c3a251b3d9f2aaa62f3a49"
server_key_iv: "c75b468f6b9fe3b840b5b0d581c65eba"
server_key_version: "kuduclusterkey@0"
I20260619 02:42:03.643621 18298 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:42:03.655352 18313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:03.655570 18298 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 02:42:03.655668 18298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "bc6a5cfb7f254f74a8a1e7c431318d95"
format_stamp: "Formatted at 2026-06-19 02:42:03 on dist-test-slave-zpbg"
server_key: "f0978ec003c3a251b3d9f2aaa62f3a49"
server_key_iv: "c75b468f6b9fe3b840b5b0d581c65eba"
server_key_version: "kuduclusterkey@0"
I20260619 02:42:03.655789 18298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:42:03.674446 18298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:42:03.676328 18298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:42:03.676493 18298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:42:03.680689 18298 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.126:38663
I20260619 02:42:03.680729 18365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.126:38663 every 8 connection(s)
I20260619 02:42:03.681068 18298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:42:03.681676 18366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:42:03.683975 18366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc6a5cfb7f254f74a8a1e7c431318d95: Bootstrap starting.
I20260619 02:42:03.684564 18366 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bc6a5cfb7f254f74a8a1e7c431318d95: Neither blocks nor log segments found. Creating new log.
I20260619 02:42:03.684861 18366 log.cc:824] T 00000000000000000000000000000000 P bc6a5cfb7f254f74a8a1e7c431318d95: Log is configured to *not* fsync() on all Append() calls
I20260619 02:42:03.685530 18366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc6a5cfb7f254f74a8a1e7c431318d95: No bootstrap required, opened a new log
I20260619 02:42:03.685599 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 18298
I20260619 02:42:03.685714 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/master-0/wal/instance
I20260619 02:42:03.685913 6085 external_mini_cluster.cc:1428] Setting key dabda4ea29e9887b99f3d8808c051063
I20260619 02:42:03.688234 18366 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc6a5cfb7f254f74a8a1e7c431318d95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc6a5cfb7f254f74a8a1e7c431318d95" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 38663 } }
I20260619 02:42:03.688424 18366 raft_consensus.cc:374] T 00000000000000000000000000000000 P bc6a5cfb7f254f74a8a1e7c431318d95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:42:03.688463 18366 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc6a5cfb7f254f74a8a1e7c431318d95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc6a5cfb7f254f74a8a1e7c431318d95, State: Initialized, Role: FOLLOWER
I20260619 02:42:03.688616 18366 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc6a5cfb7f254f74a8a1e7c431318d95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc6a5cfb7f254f74a8a1e7c431318d95" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 38663 } }
I20260619 02:42:03.688678 18366 raft_consensus.cc:388] T 00000000000000000000000000000000 P bc6a5cfb7f254f74a8a1e7c431318d95 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:42:03.688700 18366 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc6a5cfb7f254f74a8a1e7c431318d95 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:42:03.688763 18366 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc6a5cfb7f254f74a8a1e7c431318d95 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:42:03.689495 18366 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc6a5cfb7f254f74a8a1e7c431318d95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc6a5cfb7f254f74a8a1e7c431318d95" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 38663 } }
I20260619 02:42:03.689599 18366 leader_election.cc:302] T 00000000000000000000000000000000 P bc6a5cfb7f254f74a8a1e7c431318d95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bc6a5cfb7f254f74a8a1e7c431318d95; no voters:
I20260619 02:42:03.689744 18366 leader_election.cc:288] T 00000000000000000000000000000000 P bc6a5cfb7f254f74a8a1e7c431318d95 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:42:03.689824 18371 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc6a5cfb7f254f74a8a1e7c431318d95 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:42:03.689980 18366 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc6a5cfb7f254f74a8a1e7c431318d95 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:42:03.689990 18371 raft_consensus.cc:686] T 00000000000000000000000000000000 P bc6a5cfb7f254f74a8a1e7c431318d95 [term 1 LEADER]: Becoming Leader. State: Replica: bc6a5cfb7f254f74a8a1e7c431318d95, State: Running, Role: LEADER
I20260619 02:42:03.690183 18371 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc6a5cfb7f254f74a8a1e7c431318d95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc6a5cfb7f254f74a8a1e7c431318d95" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 38663 } }
I20260619 02:42:03.690526 18372 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc6a5cfb7f254f74a8a1e7c431318d95 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bc6a5cfb7f254f74a8a1e7c431318d95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc6a5cfb7f254f74a8a1e7c431318d95" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 38663 } } }
I20260619 02:42:03.690600 18372 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc6a5cfb7f254f74a8a1e7c431318d95 [sys.catalog]: This master's current role is: LEADER
I20260619 02:42:03.690850 18373 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc6a5cfb7f254f74a8a1e7c431318d95 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc6a5cfb7f254f74a8a1e7c431318d95. Latest consensus state: current_term: 1 leader_uuid: "bc6a5cfb7f254f74a8a1e7c431318d95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc6a5cfb7f254f74a8a1e7c431318d95" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 38663 } } }
I20260619 02:42:03.690932 18373 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc6a5cfb7f254f74a8a1e7c431318d95 [sys.catalog]: This master's current role is: LEADER
I20260619 02:42:03.690833 18379 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:42:03.691855 18379 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:42:03.692169 18366 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
Jun 19 02:42:03 dist-test-slave-zpbg krb5kdc[17854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836923, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:42:03.697419 18379 catalog_manager.cc:1347] Generated new cluster ID: 5767a3c52cb6487a9db3f6d0e0fc796e
I20260619 02:42:03.697562 18379 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:42:03.701217 18370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:03.686481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43824 (local address 127.5.241.126:38663)
0619 02:42:03.686685 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:03.686694 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:03.686727 (+ 33us) server_negotiation.cc:407] Connection header received
0619 02:42:03.688205 (+ 1478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:03.688214 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:03.688279 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:03.688426 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:42:03.689158 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:03.690586 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:03.691660 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:03.692311 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:03.694406 (+ 2095us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:03.695963 (+ 1557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:42:03.695970 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:42:03.695971 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:42:03.695982 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:42:03.699022 (+ 3040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:03.699462 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:03.699465 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:03.699466 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:03.699579 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:03.699805 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:03.699806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:03.699806 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:03.700201 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:42:03.700393 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:03.700477 (+ 84us) server_negotiation.cc:299] Negotiation successful
0619 02:42:03.700932 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":94,"threads_started":1}
I20260619 02:42:03.706705 18379 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:42:03.707229 18379 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:42:03.723186 18379 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bc6a5cfb7f254f74a8a1e7c431318d95: Generated new TSK 0
I20260619 02:42:03.723397 18379 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.241.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab.
Entry for principal kudu/127.5.241.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab.
I20260619 02:42:04.385861 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.241.84:34815/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.65:0
--local_ip_for_outbound_sockets=127.5.241.65
--webserver_interface=127.5.241.65
--webserver_port=0
--tserver_master_addrs=127.5.241.126:38663
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab
--principal=kudu/127.5.241.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:44677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:42:04.486825 18411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:42:04.487107 18411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:42:04.487145 18411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:42:04.488857 18411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:42:04.488920 18411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:42:04.488961 18411 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.65
I20260619 02:42:04.490756 18411 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:44677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.5.241.84:34815/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab
--principal=kudu/127.5.241.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.241.65
--webserver_port=0
--tserver_master_addrs=127.5.241.126:38663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18411
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.65
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:42:04.491101 18411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:42:04.491351 18411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:42:04.497717 18417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:42:04.497850 18411 server_base.cc:1034] running on GCE node
W20260619 02:42:04.497560 18416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:04.498126 18419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:42:04.498375 18411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:42:04.498657 18411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:42:04.499876 18411 hybrid_clock.cc:648] HybridClock initialized: now 1781836924499859 us; error 32 us; skew 500 ppm
Jun 19 02:42:04 dist-test-slave-zpbg krb5kdc[17854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836924, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:42:04.503002 18411 init.cc:377] Logged in from keytab as kudu/127.5.241.65@KRBTEST.COM (short username kudu)
I20260619 02:42:04.503568 18411 webserver.cc:460] Webserver started at http://127.5.241.65:41543/ using document root <none> and password file <none>
I20260619 02:42:04.503834 18411 fs_manager.cc:359] Metadata directory not provided
I20260619 02:42:04.503909 18411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:42:04.504070 18411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 02:42:04 dist-test-slave-zpbg krb5kdc[17854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836924, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.65@KRBTEST.COM for HTTP/127.5.241.84@KRBTEST.COM
I20260619 02:42:04.519448 18411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-0/data/instance:
uuid: "ee0f5d3172914c5e87a2fa76d6faf9fc"
format_stamp: "Formatted at 2026-06-19 02:42:04 on dist-test-slave-zpbg"
server_key: "8dfe82f94f6b007cfbf9b951bc2b44b1"
server_key_iv: "d1efbcb5e4246ffbb76ed9dc2dbde196"
server_key_version: "kuduclusterkey@0"
I20260619 02:42:04.519935 18411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-0/wal/instance:
uuid: "ee0f5d3172914c5e87a2fa76d6faf9fc"
format_stamp: "Formatted at 2026-06-19 02:42:04 on dist-test-slave-zpbg"
server_key: "8dfe82f94f6b007cfbf9b951bc2b44b1"
server_key_iv: "d1efbcb5e4246ffbb76ed9dc2dbde196"
server_key_version: "kuduclusterkey@0"
I20260619 02:42:04.521817 18411 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260619 02:42:04.539106 18426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:04.539381 18411 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.001s
I20260619 02:42:04.539492 18411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-0/wal
uuid: "ee0f5d3172914c5e87a2fa76d6faf9fc"
format_stamp: "Formatted at 2026-06-19 02:42:04 on dist-test-slave-zpbg"
server_key: "8dfe82f94f6b007cfbf9b951bc2b44b1"
server_key_iv: "d1efbcb5e4246ffbb76ed9dc2dbde196"
server_key_version: "kuduclusterkey@0"
I20260619 02:42:04.539606 18411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:42:04.558838 18411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:42:04.561354 18411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:42:04.561519 18411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:42:04.561784 18411 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:42:04.562184 18411 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:42:04.562234 18411 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:04.562265 18411 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:42:04.562323 18411 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:04.569767 18411 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.65:45197
I20260619 02:42:04.569862 18541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.65:45197 every 8 connection(s)
I20260619 02:42:04.570217 18411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
I20260619 02:42:04.572057 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 18411
I20260619 02:42:04.572163 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-0/wal/instance
I20260619 02:42:04.572346 6085 external_mini_cluster.cc:1428] Setting key a7d4a8d365412a56d1d3937b96016e9b
Jun 19 02:42:04 dist-test-slave-zpbg krb5kdc[17854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836924, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.65@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:42:04.581234 18545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:04.571021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:36449 (local address 127.5.241.126:38663)
0619 02:42:04.571270 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:04.571273 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:04.571308 (+ 35us) server_negotiation.cc:407] Connection header received
0619 02:42:04.571803 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:04.571805 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:04.571821 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:04.571910 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:42:04.572721 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:04.573487 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:04.574880 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:04.575167 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:04.575245 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:04.577897 (+ 2652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:42:04.577898 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:42:04.577899 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:42:04.577907 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:42:04.579688 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:04.580150 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:04.580151 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:04.580152 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:04.580222 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:04.580488 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:04.580489 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:04.580490 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:04.580621 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:42:04.580700 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:04.581035 (+ 335us) server_negotiation.cc:299] Negotiation successful
0619 02:42:04.581129 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":115,"threads_started":1}
WARNING: no policy specified for kudu/127.5.241.66@KRBTEST.COM; defaulting to no policy
I20260619 02:42:04.581741 18544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:04.570997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:38663 (local address 127.5.241.65:36449)
0619 02:42:04.571181 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:04.571234 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:04.571692 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:04.572015 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:04.572020 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:04.572281 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:42:04.572616 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:04.572623 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:04.573594 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:04.573595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:04.574807 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:04.574810 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:04.575279 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:04.575280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:04.575408 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:04.576092 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:42:04.576103 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:42:04.577773 (+ 1670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:42:04.579798 (+ 2025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:04.579806 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:04.579809 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:04.580054 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:04.580291 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:04.580292 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:04.580292 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:04.580417 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:04.580690 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:42:04.580696 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:42:04.580872 (+ 176us) client_negotiation.cc:769] Sending connection context
0619 02:42:04.581329 (+ 457us) client_negotiation.cc:240] Negotiation successful
0619 02:42:04.581502 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":89,"threads_started":1}
I20260619 02:42:04.585039 18542 heartbeater.cc:347] Connected to a master server at 127.5.241.126:38663
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.66@KRBTEST.COM" created.
I20260619 02:42:04.585247 18542 heartbeater.cc:464] Registering TS with master...
I20260619 02:42:04.587001 18542 heartbeater.cc:511] Master 127.5.241.126:38663 requested a full tablet report, sending...
I20260619 02:42:04.587697 18330 ts_manager.cc:194] Registered new tserver with Master: ee0f5d3172914c5e87a2fa76d6faf9fc (127.5.241.65:45197)
I20260619 02:42:04.588624 18330 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.65@KRBTEST.COM'} at 127.5.241.65:36449
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab.
Entry for principal kudu/127.5.241.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab.
I20260619 02:42:04.606555 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.241.84:34815/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.66:0
--local_ip_for_outbound_sockets=127.5.241.66
--webserver_interface=127.5.241.66
--webserver_port=0
--tserver_master_addrs=127.5.241.126:38663
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab
--principal=kudu/127.5.241.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:44677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 02:42:04 dist-test-slave-zpbg krb5kdc[17854](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781836924, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260619 02:42:04.725987 18548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:42:04.726276 18548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:42:04.726320 18548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:42:04.728082 18548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:42:04.728174 18548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:42:04.728235 18548 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.66
I20260619 02:42:04.730393 18548 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:44677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.5.241.84:34815/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab
--principal=kudu/127.5.241.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.241.66
--webserver_port=0
--tserver_master_addrs=127.5.241.126:38663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18548
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.66
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:42:04.730759 18548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:42:04.731073 18548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:42:04.737572 18555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:04.737591 18556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:04.738013 18558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:42:04.738688 18548 server_base.cc:1034] running on GCE node
I20260619 02:42:04.738968 18548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:42:04.739238 18548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:42:04.740417 18548 hybrid_clock.cc:648] HybridClock initialized: now 1781836924740400 us; error 41 us; skew 500 ppm
Jun 19 02:42:04 dist-test-slave-zpbg krb5kdc[17854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836924, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:42:04.743791 18548 init.cc:377] Logged in from keytab as kudu/127.5.241.66@KRBTEST.COM (short username kudu)
I20260619 02:42:04.744323 18548 webserver.cc:460] Webserver started at http://127.5.241.66:39699/ using document root <none> and password file <none>
I20260619 02:42:04.744570 18548 fs_manager.cc:359] Metadata directory not provided
I20260619 02:42:04.744625 18548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:42:04.744760 18548 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 02:42:04 dist-test-slave-zpbg krb5kdc[17854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836924, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.66@KRBTEST.COM for HTTP/127.5.241.84@KRBTEST.COM
I20260619 02:42:04.766705 18548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-1/data/instance:
uuid: "73552824d4554814928278b51d067fa5"
format_stamp: "Formatted at 2026-06-19 02:42:04 on dist-test-slave-zpbg"
server_key: "ee622b8e731274bcd1a6a8f68e7fd9b2"
server_key_iv: "550305f1be8bd01fec0e2ffbcbb8f859"
server_key_version: "kuduclusterkey@0"
I20260619 02:42:04.767207 18548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-1/wal/instance:
uuid: "73552824d4554814928278b51d067fa5"
format_stamp: "Formatted at 2026-06-19 02:42:04 on dist-test-slave-zpbg"
server_key: "ee622b8e731274bcd1a6a8f68e7fd9b2"
server_key_iv: "550305f1be8bd01fec0e2ffbcbb8f859"
server_key_version: "kuduclusterkey@0"
I20260619 02:42:04.769203 18548 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260619 02:42:04.780992 18565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:04.781239 18548 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:42:04.781338 18548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-1/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-1/wal
uuid: "73552824d4554814928278b51d067fa5"
format_stamp: "Formatted at 2026-06-19 02:42:04 on dist-test-slave-zpbg"
server_key: "ee622b8e731274bcd1a6a8f68e7fd9b2"
server_key_iv: "550305f1be8bd01fec0e2ffbcbb8f859"
server_key_version: "kuduclusterkey@0"
I20260619 02:42:04.781457 18548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:42:04.793922 18548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:42:04.796463 18548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:42:04.796674 18548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:42:04.796953 18548 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:42:04.797356 18548 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:42:04.797441 18548 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:04.797511 18548 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:42:04.797569 18548 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:04.804461 18548 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.66:45087
I20260619 02:42:04.804967 18548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
I20260619 02:42:04.807514 18678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.66:45087 every 8 connection(s)
I20260619 02:42:04.807657 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 18548
I20260619 02:42:04.807768 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-1/wal/instance
I20260619 02:42:04.808002 6085 external_mini_cluster.cc:1428] Setting key c44801a459385e96fb8c82dca455f398
Jun 19 02:42:04 dist-test-slave-zpbg krb5kdc[17854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836924, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.66@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:42:04.818941 18545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:04.807453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:52329 (local address 127.5.241.126:38663)
0619 02:42:04.808206 (+ 753us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:04.808208 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:04.808223 (+ 15us) server_negotiation.cc:407] Connection header received
0619 02:42:04.809294 (+ 1071us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:04.809309 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:04.809326 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:04.809362 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:42:04.810211 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:04.811050 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:04.812567 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:04.812867 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:04.812916 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:04.815567 (+ 2651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:42:04.815568 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:42:04.815568 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:42:04.815576 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:42:04.817337 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:04.817815 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:04.817816 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:04.817817 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:04.817875 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:04.818149 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:04.818150 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:04.818150 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:04.818286 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:42:04.818340 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:04.818704 (+ 364us) server_negotiation.cc:299] Negotiation successful
0619 02:42:04.818838 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":720}
I20260619 02:42:04.819706 18681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:04.807136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:38663 (local address 127.5.241.66:52329)
0619 02:42:04.808058 (+ 922us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:04.808101 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:04.809169 (+ 1068us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:04.809550 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:04.809554 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:04.809803 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:42:04.810094 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:04.810101 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:04.811168 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:04.811169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:04.812462 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:04.812464 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:04.813024 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:04.813025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:04.813160 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:04.813782 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:42:04.813792 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:42:04.815457 (+ 1665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:42:04.817435 (+ 1978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:04.817441 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:04.817444 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:04.817727 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:04.817946 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:04.817947 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:04.817947 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:04.818074 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:04.818388 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:42:04.818393 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:42:04.818549 (+ 156us) client_negotiation.cc:769] Sending connection context
0619 02:42:04.819308 (+ 759us) client_negotiation.cc:240] Negotiation successful
0619 02:42:04.819471 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":883,"thread_start_us":83,"threads_started":1}
I20260619 02:42:04.820381 18679 heartbeater.cc:347] Connected to a master server at 127.5.241.126:38663
I20260619 02:42:04.820474 18679 heartbeater.cc:464] Registering TS with master...
I20260619 02:42:04.820674 18679 heartbeater.cc:511] Master 127.5.241.126:38663 requested a full tablet report, sending...
I20260619 02:42:04.821141 18330 ts_manager.cc:194] Registered new tserver with Master: 73552824d4554814928278b51d067fa5 (127.5.241.66:45087)
WARNING: no policy specified for kudu/127.5.241.67@KRBTEST.COM; defaulting to no policy
I20260619 02:42:04.821650 18330 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.66@KRBTEST.COM'} at 127.5.241.66:52329
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab.
Entry for principal kudu/127.5.241.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab.
I20260619 02:42:04.851188 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.241.84:34815/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.67:0
--local_ip_for_outbound_sockets=127.5.241.67
--webserver_interface=127.5.241.67
--webserver_port=0
--tserver_master_addrs=127.5.241.126:38663
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab
--principal=kudu/127.5.241.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:44677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:42:04.964761 18685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:42:04.965127 18685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:42:04.965215 18685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:42:04.967022 18685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:42:04.967142 18685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:42:04.967236 18685 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.67
I20260619 02:42:04.969393 18685 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:44677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.5.241.84:34815/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab
--principal=kudu/127.5.241.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.241.67
--webserver_port=0
--tserver_master_addrs=127.5.241.126:38663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.67
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:42:04.969949 18685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:42:04.970245 18685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:42:04.984632 18691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:04.984658 18690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:04.986851 18693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:42:04.987146 18685 server_base.cc:1034] running on GCE node
I20260619 02:42:04.987402 18685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:42:04.987726 18685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:42:04.995103 18685 hybrid_clock.cc:648] HybridClock initialized: now 1781836924995079 us; error 57 us; skew 500 ppm
Jun 19 02:42:04 dist-test-slave-zpbg krb5kdc[17854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836924, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:42:04.998641 18685 init.cc:377] Logged in from keytab as kudu/127.5.241.67@KRBTEST.COM (short username kudu)
I20260619 02:42:04.999262 18685 webserver.cc:460] Webserver started at http://127.5.241.67:38245/ using document root <none> and password file <none>
I20260619 02:42:04.999571 18685 fs_manager.cc:359] Metadata directory not provided
I20260619 02:42:04.999680 18685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:42:04.999868 18685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 02:42:04 dist-test-slave-zpbg krb5kdc[17854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836924, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.67@KRBTEST.COM for HTTP/127.5.241.84@KRBTEST.COM
I20260619 02:42:05.019114 18685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-2/data/instance:
uuid: "1ae98af952b04b94a24e1bd94a4d8526"
format_stamp: "Formatted at 2026-06-19 02:42:05 on dist-test-slave-zpbg"
server_key: "a09f32846817c4431b7e6f51a9f6cc97"
server_key_iv: "4db4718f3af222b79dbc3eaada5821ea"
server_key_version: "kuduclusterkey@0"
I20260619 02:42:05.019537 18685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-2/wal/instance:
uuid: "1ae98af952b04b94a24e1bd94a4d8526"
format_stamp: "Formatted at 2026-06-19 02:42:05 on dist-test-slave-zpbg"
server_key: "a09f32846817c4431b7e6f51a9f6cc97"
server_key_iv: "4db4718f3af222b79dbc3eaada5821ea"
server_key_version: "kuduclusterkey@0"
I20260619 02:42:05.020972 18685 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:42:05.049046 18700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:05.049283 18685 fs_manager.cc:725] Time spent opening block manager: real 0.014s user 0.000s sys 0.000s
I20260619 02:42:05.049376 18685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-2/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-2/wal
uuid: "1ae98af952b04b94a24e1bd94a4d8526"
format_stamp: "Formatted at 2026-06-19 02:42:05 on dist-test-slave-zpbg"
server_key: "a09f32846817c4431b7e6f51a9f6cc97"
server_key_iv: "4db4718f3af222b79dbc3eaada5821ea"
server_key_version: "kuduclusterkey@0"
I20260619 02:42:05.049472 18685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:42:05.073020 18685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:42:05.075690 18685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:42:05.075919 18685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:42:05.076210 18685 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:42:05.076618 18685 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:42:05.076712 18685 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:05.076797 18685 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:42:05.076856 18685 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:05.085229 18685 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.67:40225
I20260619 02:42:05.085773 18685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
I20260619 02:42:05.086154 18813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.67:40225 every 8 connection(s)
I20260619 02:42:05.089540 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 18685
I20260619 02:42:05.089639 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-2/wal/instance
I20260619 02:42:05.089834 6085 external_mini_cluster.cc:1428] Setting key 8ab518ae423dee693154457b83dce6bd
Jun 19 02:42:05 dist-test-slave-zpbg krb5kdc[17854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836924, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.67@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:42:05.097826 18545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:05.086873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:39577 (local address 127.5.241.126:38663)
0619 02:42:05.086920 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:05.086923 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:05.087251 (+ 328us) server_negotiation.cc:407] Connection header received
0619 02:42:05.087934 (+ 683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:05.087935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:05.087949 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:05.087980 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:42:05.088844 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:05.089593 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:05.091220 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:05.091517 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:05.091565 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:05.094414 (+ 2849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:42:05.094416 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:42:05.094416 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:42:05.094424 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:42:05.096256 (+ 1832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:05.096760 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:05.096761 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:05.096762 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:05.096819 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:05.097092 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:05.097093 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:05.097093 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:05.097225 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:42:05.097268 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:05.097624 (+ 356us) server_negotiation.cc:299] Negotiation successful
0619 02:42:05.097712 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:42:05.098737 18816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:05.086988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:38663 (local address 127.5.241.67:39577)
0619 02:42:05.087125 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:05.087165 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:05.087808 (+ 643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:05.088164 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:05.088168 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:05.088443 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:42:05.088715 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:05.088722 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:05.089721 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:05.089721 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:05.091117 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:05.091120 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:05.091673 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:05.091674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:05.091825 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:05.092434 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:42:05.092444 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:42:05.094189 (+ 1745us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:42:05.096361 (+ 2172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:05.096368 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:05.096370 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:05.096658 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:05.096890 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:05.096890 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:05.096891 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:05.097016 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:05.097315 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:42:05.097321 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:42:05.097474 (+ 153us) client_negotiation.cc:769] Sending connection context
0619 02:42:05.098171 (+ 697us) client_negotiation.cc:240] Negotiation successful
0619 02:42:05.098334 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":74,"threads_started":1}
I20260619 02:42:05.099432 18814 heartbeater.cc:347] Connected to a master server at 127.5.241.126:38663
I20260619 02:42:05.099520 18814 heartbeater.cc:464] Registering TS with master...
I20260619 02:42:05.099704 18814 heartbeater.cc:511] Master 127.5.241.126:38663 requested a full tablet report, sending...
I20260619 02:42:05.100114 18330 ts_manager.cc:194] Registered new tserver with Master: 1ae98af952b04b94a24e1bd94a4d8526 (127.5.241.67:40225)
I20260619 02:42:05.100610 18330 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.67@KRBTEST.COM'} at 127.5.241.67:39577
I20260619 02:42:05.101253 6085 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:42:05.101830 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 18411
I20260619 02:42:05.108364 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 18548
I20260619 02:42:05.115500 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 18685
I20260619 02:42:05.120442 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 18298
I20260619 02:42:05.125411 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:38663
--webserver_interface=127.5.241.126
--webserver_port=44925
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:44677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:38663
--ranger_config_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.241.84:34815/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.241.84:34815/kms with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:42:05.208433 18817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:42:05.208665 18817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:42:05.208711 18817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:42:05.209853 18817 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:42:05.209920 18817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:42:05.209947 18817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:42:05.209978 18817 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:42:05.210013 18817 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:42:05.211637 18817 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:44677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.241.84:34815/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:38663
--ranger_config_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:38663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.241.126
--webserver_port=44925
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18817
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:42:05.211905 18817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:42:05.212126 18817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:42:05.217967 18822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:05.218039 18825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:05.217998 18823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:42:05.218362 18817 server_base.cc:1034] running on GCE node
I20260619 02:42:05.218600 18817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:42:05.218988 18817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:42:05.220167 18817 hybrid_clock.cc:648] HybridClock initialized: now 1781836925220154 us; error 28 us; skew 500 ppm
Jun 19 02:42:05 dist-test-slave-zpbg krb5kdc[17854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836925, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:42:05.222869 18817 init.cc:377] Logged in from keytab as kudu/127.5.241.126@KRBTEST.COM (short username kudu)
I20260619 02:42:05.223369 18817 webserver.cc:460] Webserver started at http://127.5.241.126:44925/ using document root <none> and password file <none>
I20260619 02:42:05.223634 18817 fs_manager.cc:359] Metadata directory not provided
I20260619 02:42:05.223691 18817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 02:42:05 dist-test-slave-zpbg krb5kdc[17854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836925, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for HTTP/127.5.241.84@KRBTEST.COM
I20260619 02:42:05.304257 18817 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:42:05.305018 18833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:05.305177 18817 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:42:05.305287 18817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "bc6a5cfb7f254f74a8a1e7c431318d95"
format_stamp: "Formatted at 2026-06-19 02:42:03 on dist-test-slave-zpbg"
server_key: "f0978ec003c3a251b3d9f2aaa62f3a49"
server_key_iv: "c75b468f6b9fe3b840b5b0d581c65eba"
server_key_version: "kuduclusterkey@0"
I20260619 02:42:05.305658 18817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:42:05.334101 18817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:42:05.336315 18817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:42:05.336472 18817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:42:05.341390 18817 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.126:38663
I20260619 02:42:05.341418 18885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.126:38663 every 8 connection(s)
I20260619 02:42:05.341797 18817 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:42:05.342495 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 18817
I20260619 02:42:05.342824 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.241.84:34815/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.65:45197
--local_ip_for_outbound_sockets=127.5.241.65
--tserver_master_addrs=127.5.241.126:38663
--webserver_port=41543
--webserver_interface=127.5.241.65
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab
--principal=kudu/127.5.241.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:44677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.241.84:34815/kms with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:42:05.343279 18886 sys_catalog.cc:263] Verifying existing consensus state
I20260619 02:42:05.344307 18886 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc6a5cfb7f254f74a8a1e7c431318d95: Bootstrap starting.
I20260619 02:42:05.355326 18886 log.cc:824] T 00000000000000000000000000000000 P bc6a5cfb7f254f74a8a1e7c431318d95: Log is configured to *not* fsync() on all Append() calls
I20260619 02:42:05.357703 18886 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc6a5cfb7f254f74a8a1e7c431318d95: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:42:05.358034 18886 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc6a5cfb7f254f74a8a1e7c431318d95: Bootstrap complete.
I20260619 02:42:05.361403 18886 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc6a5cfb7f254f74a8a1e7c431318d95 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc6a5cfb7f254f74a8a1e7c431318d95" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 38663 } }
I20260619 02:42:05.361729 18886 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc6a5cfb7f254f74a8a1e7c431318d95 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc6a5cfb7f254f74a8a1e7c431318d95, State: Initialized, Role: FOLLOWER
I20260619 02:42:05.361939 18886 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc6a5cfb7f254f74a8a1e7c431318d95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc6a5cfb7f254f74a8a1e7c431318d95" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 38663 } }
I20260619 02:42:05.362005 18886 raft_consensus.cc:388] T 00000000000000000000000000000000 P bc6a5cfb7f254f74a8a1e7c431318d95 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:42:05.362042 18886 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc6a5cfb7f254f74a8a1e7c431318d95 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:42:05.362087 18886 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc6a5cfb7f254f74a8a1e7c431318d95 [term 1 FOLLOWER]: Advancing to term 2
I20260619 02:42:05.363551 18886 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc6a5cfb7f254f74a8a1e7c431318d95 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc6a5cfb7f254f74a8a1e7c431318d95" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 38663 } }
I20260619 02:42:05.363667 18886 leader_election.cc:302] T 00000000000000000000000000000000 P bc6a5cfb7f254f74a8a1e7c431318d95 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bc6a5cfb7f254f74a8a1e7c431318d95; no voters:
I20260619 02:42:05.363839 18886 leader_election.cc:288] T 00000000000000000000000000000000 P bc6a5cfb7f254f74a8a1e7c431318d95 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 02:42:05.364039 18890 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc6a5cfb7f254f74a8a1e7c431318d95 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 02:42:05.364078 18886 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc6a5cfb7f254f74a8a1e7c431318d95 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:42:05.364298 18890 raft_consensus.cc:686] T 00000000000000000000000000000000 P bc6a5cfb7f254f74a8a1e7c431318d95 [term 2 LEADER]: Becoming Leader. State: Replica: bc6a5cfb7f254f74a8a1e7c431318d95, State: Running, Role: LEADER
I20260619 02:42:05.364462 18890 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc6a5cfb7f254f74a8a1e7c431318d95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc6a5cfb7f254f74a8a1e7c431318d95" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 38663 } }
I20260619 02:42:05.364790 18890 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc6a5cfb7f254f74a8a1e7c431318d95 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "bc6a5cfb7f254f74a8a1e7c431318d95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc6a5cfb7f254f74a8a1e7c431318d95" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 38663 } } }
I20260619 02:42:05.364918 18890 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc6a5cfb7f254f74a8a1e7c431318d95 [sys.catalog]: This master's current role is: LEADER
I20260619 02:42:05.365087 18890 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc6a5cfb7f254f74a8a1e7c431318d95 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc6a5cfb7f254f74a8a1e7c431318d95. Latest consensus state: current_term: 2 leader_uuid: "bc6a5cfb7f254f74a8a1e7c431318d95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc6a5cfb7f254f74a8a1e7c431318d95" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 38663 } } }
I20260619 02:42:05.365192 18890 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc6a5cfb7f254f74a8a1e7c431318d95 [sys.catalog]: This master's current role is: LEADER
I20260619 02:42:05.365343 18899 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:42:05.365590 18886 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 02:42:05.365957 18899 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:42:05.366096 18899 catalog_manager.cc:1259] Loaded cluster ID: 5767a3c52cb6487a9db3f6d0e0fc796e
I20260619 02:42:05.366132 18899 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:42:05.369809 18899 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:42:05.370422 18899 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bc6a5cfb7f254f74a8a1e7c431318d95: Loaded TSK: 0
I20260619 02:42:05.370831 18899 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 02:42:05.459771 18888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:42:05.460054 18888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:42:05.460093 18888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:42:05.461864 18888 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:42:05.461926 18888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:42:05.461967 18888 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.65
I20260619 02:42:05.464175 18888 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:44677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.241.84:34815/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab
--principal=kudu/127.5.241.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.65:45197
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.241.65
--webserver_port=41543
--tserver_master_addrs=127.5.241.126:38663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18888
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.65
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:42:05.464425 18888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:42:05.464673 18888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:42:05.470901 18923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:05.471062 18926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:05.470887 18924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:42:05.471931 18888 server_base.cc:1034] running on GCE node
I20260619 02:42:05.472172 18888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:42:05.472412 18888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:42:05.476898 18888 hybrid_clock.cc:648] HybridClock initialized: now 1781836925476865 us; error 48 us; skew 500 ppm
Jun 19 02:42:05 dist-test-slave-zpbg krb5kdc[17854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836925, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:42:05.480041 18888 init.cc:377] Logged in from keytab as kudu/127.5.241.65@KRBTEST.COM (short username kudu)
I20260619 02:42:05.480587 18888 webserver.cc:460] Webserver started at http://127.5.241.65:41543/ using document root <none> and password file <none>
I20260619 02:42:05.480847 18888 fs_manager.cc:359] Metadata directory not provided
I20260619 02:42:05.480932 18888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 02:42:05 dist-test-slave-zpbg krb5kdc[17854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836925, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.65@KRBTEST.COM for HTTP/127.5.241.84@KRBTEST.COM
I20260619 02:42:05.501735 18888 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.000s
I20260619 02:42:05.503239 18934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:05.503472 18888 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:42:05.503562 18888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-0/wal
uuid: "ee0f5d3172914c5e87a2fa76d6faf9fc"
format_stamp: "Formatted at 2026-06-19 02:42:04 on dist-test-slave-zpbg"
server_key: "8dfe82f94f6b007cfbf9b951bc2b44b1"
server_key_iv: "d1efbcb5e4246ffbb76ed9dc2dbde196"
server_key_version: "kuduclusterkey@0"
I20260619 02:42:05.504056 18888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:42:05.553634 18888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:42:05.555830 18888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:42:05.555989 18888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:42:05.556209 18888 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:42:05.556600 18888 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:42:05.556658 18888 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:05.556690 18888 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:42:05.556713 18888 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:05.564021 18888 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.65:45197
I20260619 02:42:05.564152 19047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.65:45197 every 8 connection(s)
I20260619 02:42:05.564543 18888 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
I20260619 02:42:05.570672 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 18888
I20260619 02:42:05.570988 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.241.84:34815/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.66:45087
--local_ip_for_outbound_sockets=127.5.241.66
--tserver_master_addrs=127.5.241.126:38663
--webserver_port=39699
--webserver_interface=127.5.241.66
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab
--principal=kudu/127.5.241.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:44677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.241.84:34815/kms with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 02:42:05 dist-test-slave-zpbg krb5kdc[17854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836925, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.65@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:42:05.582615 19050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:05.565418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:38663 (local address 127.5.241.65:58769)
0619 02:42:05.565644 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:05.565693 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:05.566148 (+ 455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:05.567427 (+ 1279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:05.567432 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:05.567722 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:42:05.568005 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:05.568011 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:05.569395 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:05.569396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:05.572614 (+ 3218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:05.572617 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:05.573517 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:05.573519 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:05.573665 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:05.574894 (+ 1229us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:42:05.574904 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:42:05.578313 (+ 3409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:42:05.580687 (+ 2374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:05.580694 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:05.580696 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:05.580998 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:05.581314 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:05.581315 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:05.581316 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:05.581444 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:05.581931 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:42:05.581936 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:42:05.582088 (+ 152us) client_negotiation.cc:769] Sending connection context
0619 02:42:05.582184 (+ 96us) client_negotiation.cc:240] Negotiation successful
0619 02:42:05.582359 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":95,"threads_started":1}
I20260619 02:42:05.583382 19051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:05.565480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:58769 (local address 127.5.241.126:38663)
0619 02:42:05.566919 (+ 1439us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:05.566928 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:05.566974 (+ 46us) server_negotiation.cc:407] Connection header received
0619 02:42:05.567123 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:05.567131 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:05.567190 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:05.567349 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:42:05.568137 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:05.569283 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:05.572701 (+ 3418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:05.573380 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:05.573960 (+ 580us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:05.578450 (+ 4490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:42:05.578455 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:42:05.578455 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:42:05.578469 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:42:05.580583 (+ 2114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:05.581102 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:05.581105 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:05.581106 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:05.581216 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:05.581528 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:05.581529 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:05.581529 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:05.581836 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:42:05.582967 (+ 1131us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:05.583043 (+ 76us) server_negotiation.cc:299] Negotiation successful
0619 02:42:05.583169 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1374,"thread_start_us":115,"threads_started":1}
I20260619 02:42:05.584314 19048 heartbeater.cc:347] Connected to a master server at 127.5.241.126:38663
I20260619 02:42:05.584448 19048 heartbeater.cc:464] Registering TS with master...
I20260619 02:42:05.584657 19048 heartbeater.cc:511] Master 127.5.241.126:38663 requested a full tablet report, sending...
W20260619 02:42:05.585520 19048 heartbeater.cc:644] Failed to heartbeat to 127.5.241.126:38663 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 02:42:05.585592 19048 heartbeater.cc:464] Registering TS with master...
I20260619 02:42:05.585655 19048 heartbeater.cc:511] Master 127.5.241.126:38663 requested a full tablet report, sending...
I20260619 02:42:05.586069 19048 heartbeater.cc:464] Registering TS with master...
I20260619 02:42:05.586143 19048 heartbeater.cc:511] Master 127.5.241.126:38663 requested a full tablet report, sending...
W20260619 02:42:05.586503 19048 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 02:42:05.690852 19052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:42:05.691157 19052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:42:05.691227 19052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:42:05.692955 19052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:42:05.693044 19052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:42:05.693100 19052 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.66
I20260619 02:42:05.695241 19052 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:44677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.241.84:34815/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab
--principal=kudu/127.5.241.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.66:45087
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.241.66
--webserver_port=39699
--tserver_master_addrs=127.5.241.126:38663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19052
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.66
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:42:05.695629 19052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:42:05.695915 19052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:42:05.703120 19058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:05.703111 19060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:42:05.703359 19052 server_base.cc:1034] running on GCE node
W20260619 02:42:05.703141 19057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:42:05.703668 19052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:42:05.703883 19052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:42:05.705044 19052 hybrid_clock.cc:648] HybridClock initialized: now 1781836925705032 us; error 31 us; skew 500 ppm
Jun 19 02:42:05 dist-test-slave-zpbg krb5kdc[17854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836925, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:42:05.707829 19052 init.cc:377] Logged in from keytab as kudu/127.5.241.66@KRBTEST.COM (short username kudu)
I20260619 02:42:05.708298 19052 webserver.cc:460] Webserver started at http://127.5.241.66:39699/ using document root <none> and password file <none>
I20260619 02:42:05.708536 19052 fs_manager.cc:359] Metadata directory not provided
I20260619 02:42:05.708616 19052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 02:42:05 dist-test-slave-zpbg krb5kdc[17854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836925, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.66@KRBTEST.COM for HTTP/127.5.241.84@KRBTEST.COM
I20260619 02:42:05.723259 19052 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:42:05.727685 19068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:05.727928 19052 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.000s
I20260619 02:42:05.728034 19052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-1/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-1/wal
uuid: "73552824d4554814928278b51d067fa5"
format_stamp: "Formatted at 2026-06-19 02:42:04 on dist-test-slave-zpbg"
server_key: "ee622b8e731274bcd1a6a8f68e7fd9b2"
server_key_iv: "550305f1be8bd01fec0e2ffbcbb8f859"
server_key_version: "kuduclusterkey@0"
I20260619 02:42:05.728547 19052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:42:05.748945 19052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:42:05.751392 19052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:42:05.751600 19052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:42:05.751869 19052 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:42:05.752269 19052 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:42:05.752315 19052 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:05.752352 19052 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:42:05.752425 19052 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:05.759907 19052 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.66:45087
I20260619 02:42:05.760362 19052 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
I20260619 02:42:05.760872 19181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.66:45087 every 8 connection(s)
Jun 19 02:42:05 dist-test-slave-zpbg krb5kdc[17854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836925, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.66@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:42:05.770216 19051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:05.761179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:54737 (local address 127.5.241.126:38663)
0619 02:42:05.761236 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:05.761239 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:05.761489 (+ 250us) server_negotiation.cc:407] Connection header received
0619 02:42:05.762116 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:05.762117 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:05.762133 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:05.762185 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:42:05.762755 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:05.763291 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:05.764505 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:05.764723 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:05.764794 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:05.767342 (+ 2548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:42:05.767343 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:42:05.767344 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:42:05.767352 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:42:05.768779 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:05.769212 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:05.769213 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:05.769214 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:05.769278 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:05.769534 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:05.769535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:05.769535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:05.769652 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:42:05.769729 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:05.770009 (+ 280us) server_negotiation.cc:299] Negotiation successful
0619 02:42:05.770095 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 02:42:05.770243 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 19052
I20260619 02:42:05.770524 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.241.84:34815/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.67:40225
--local_ip_for_outbound_sockets=127.5.241.67
--tserver_master_addrs=127.5.241.126:38663
--webserver_port=38245
--webserver_interface=127.5.241.67
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab
--principal=kudu/127.5.241.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:44677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.241.84:34815/kms with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:42:05.771464 19184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:05.761164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:38663 (local address 127.5.241.66:54737)
0619 02:42:05.761345 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:05.761392 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:05.761996 (+ 604us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:05.762253 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:05.762257 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:05.762455 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:42:05.762646 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:05.762650 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:05.763387 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:05.763388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:05.764407 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:05.764410 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:05.764833 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:05.764834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:05.764992 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:05.765826 (+ 834us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:42:05.765834 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:42:05.767240 (+ 1406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:42:05.768880 (+ 1640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:05.768887 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:05.768890 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:05.769120 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:05.769346 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:05.769347 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:05.769347 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:05.769466 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:05.769720 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:42:05.769727 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:42:05.769878 (+ 151us) client_negotiation.cc:769] Sending connection context
0619 02:42:05.769991 (+ 113us) client_negotiation.cc:240] Negotiation successful
0619 02:42:05.771166 (+ 1175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":90,"threads_started":1}
I20260619 02:42:05.772222 19182 heartbeater.cc:347] Connected to a master server at 127.5.241.126:38663
I20260619 02:42:05.772344 19182 heartbeater.cc:464] Registering TS with master...
I20260619 02:42:05.772583 19182 heartbeater.cc:511] Master 127.5.241.126:38663 requested a full tablet report, sending...
W20260619 02:42:05.773422 19182 heartbeater.cc:644] Failed to heartbeat to 127.5.241.126:38663 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 02:42:05.773497 19182 heartbeater.cc:464] Registering TS with master...
I20260619 02:42:05.773566 19182 heartbeater.cc:511] Master 127.5.241.126:38663 requested a full tablet report, sending...
I20260619 02:42:05.773933 19182 heartbeater.cc:464] Registering TS with master...
I20260619 02:42:05.774066 19182 heartbeater.cc:511] Master 127.5.241.126:38663 requested a full tablet report, sending...
W20260619 02:42:05.774485 19182 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 02:42:05.867759 19185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:42:05.867981 19185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:42:05.868023 19185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:42:05.869237 19185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:42:05.869309 19185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:42:05.869355 19185 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.67
I20260619 02:42:05.871719 19185 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:44677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.241.84:34815/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab
--principal=kudu/127.5.241.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.67:40225
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.241.67
--webserver_port=38245
--tserver_master_addrs=127.5.241.126:38663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19185
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.67
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:42:05.872304 19185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:42:05.872640 19185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:42:05.882843 19192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:05.882932 19194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:05.883342 19191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:42:05.884167 19185 server_base.cc:1034] running on GCE node
I20260619 02:42:05.884459 19185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:42:05.884743 19185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:42:05.885921 19185 hybrid_clock.cc:648] HybridClock initialized: now 1781836925885893 us; error 41 us; skew 500 ppm
Jun 19 02:42:05 dist-test-slave-zpbg krb5kdc[17854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836925, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:42:05.889032 19185 init.cc:377] Logged in from keytab as kudu/127.5.241.67@KRBTEST.COM (short username kudu)
I20260619 02:42:05.889536 19185 webserver.cc:460] Webserver started at http://127.5.241.67:38245/ using document root <none> and password file <none>
I20260619 02:42:05.889801 19185 fs_manager.cc:359] Metadata directory not provided
I20260619 02:42:05.889858 19185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 02:42:05 dist-test-slave-zpbg krb5kdc[17854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836925, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.67@KRBTEST.COM for HTTP/127.5.241.84@KRBTEST.COM
I20260619 02:42:05.907766 19185 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:42:05.911739 19202 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:05.912029 19185 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.000s
I20260619 02:42:05.912217 19185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-2/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-2/wal
uuid: "1ae98af952b04b94a24e1bd94a4d8526"
format_stamp: "Formatted at 2026-06-19 02:42:05 on dist-test-slave-zpbg"
server_key: "a09f32846817c4431b7e6f51a9f6cc97"
server_key_iv: "4db4718f3af222b79dbc3eaada5821ea"
server_key_version: "kuduclusterkey@0"
I20260619 02:42:05.912717 19185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:42:05.946182 19185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:42:05.948773 19185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:42:05.949000 19185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:42:05.949311 19185 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:42:05.949708 19185 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:42:05.949760 19185 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:05.949793 19185 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:42:05.949811 19185 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:05.957258 19185 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.67:40225
I20260619 02:42:05.957336 19315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.67:40225 every 8 connection(s)
I20260619 02:42:05.957922 19185 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
Jun 19 02:42:05 dist-test-slave-zpbg krb5kdc[17854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836925, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.67@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:42:05.967226 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 19185
I20260619 02:42:05.974395 19051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:05.959674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:37305 (local address 127.5.241.126:38663)
0619 02:42:05.959723 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:05.959725 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:05.960184 (+ 459us) server_negotiation.cc:407] Connection header received
0619 02:42:05.960894 (+ 710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:05.960896 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:05.960908 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:05.960988 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:42:05.961935 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:05.962466 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:05.964187 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:05.964497 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:05.964628 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:05.971311 (+ 6683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:42:05.971313 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:42:05.971314 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:42:05.971323 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:42:05.972889 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:05.973415 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:05.973416 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:05.973416 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:05.973473 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:05.973730 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:05.973730 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:05.973731 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:05.973842 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:42:05.973900 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:05.974217 (+ 317us) server_negotiation.cc:299] Negotiation successful
0619 02:42:05.974296 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 02:42:05.974614 19317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:05.959398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:38663 (local address 127.5.241.67:37305)
0619 02:42:05.960052 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:05.960096 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:05.960765 (+ 669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:05.961140 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:05.961144 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:05.961381 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:42:05.961656 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:05.961664 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:05.962668 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:05.962669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:05.964048 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:05.964051 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:05.964732 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:05.964734 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:05.964888 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:05.965565 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:42:05.965575 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:42:05.971139 (+ 5564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:42:05.972986 (+ 1847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:05.972996 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:05.972999 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:05.973325 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:05.973536 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:05.973537 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:05.973537 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:05.973668 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:05.973924 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:42:05.973929 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:42:05.974070 (+ 141us) client_negotiation.cc:769] Sending connection context
0619 02:42:05.974186 (+ 116us) client_negotiation.cc:240] Negotiation successful
0619 02:42:05.974354 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":76,"threads_started":1}
I20260619 02:42:05.975476 19316 heartbeater.cc:347] Connected to a master server at 127.5.241.126:38663
I20260619 02:42:05.975593 19316 heartbeater.cc:464] Registering TS with master...
I20260619 02:42:05.975806 19316 heartbeater.cc:511] Master 127.5.241.126:38663 requested a full tablet report, sending...
W20260619 02:42:05.976488 19316 heartbeater.cc:644] Failed to heartbeat to 127.5.241.126:38663 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 02:42:05.976575 19316 heartbeater.cc:464] Registering TS with master...
I20260619 02:42:05.976651 19316 heartbeater.cc:511] Master 127.5.241.126:38663 requested a full tablet report, sending...
I20260619 02:42:05.977099 19316 heartbeater.cc:464] Registering TS with master...
I20260619 02:42:05.977187 19316 heartbeater.cc:511] Master 127.5.241.126:38663 requested a full tablet report, sending...
W20260619 02:42:05.977699 19316 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 02:42:05.979959 19320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:05.967658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56632 (local address 127.5.241.126:38663)
0619 02:42:05.971328 (+ 3670us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:05.971331 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:05.971454 (+ 123us) server_negotiation.cc:407] Connection header received
0619 02:42:05.971551 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:05.971552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:05.971564 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:05.971915 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:42:05.973049 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:05.973819 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:05.975161 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:05.975472 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:05.975547 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:05.977125 (+ 1578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:42:05.977126 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:42:05.977126 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:42:05.977133 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:42:05.978757 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:05.979155 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:05.979156 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:05.979156 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:05.979218 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:05.979402 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:05.979403 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:05.979404 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:05.979541 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:42:05.979719 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:05.979774 (+ 55us) server_negotiation.cc:299] Negotiation successful
0619 02:42:05.979862 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3641,"thread_start_us":69,"threads_started":1}
Jun 19 02:42:06 dist-test-slave-zpbg krb5kdc[17854](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781836926, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:42:06.586664 19048 heartbeater.cc:464] Registering TS with master...
I20260619 02:42:06.586915 19048 heartbeater.cc:511] Master 127.5.241.126:38663 requested a full tablet report, sending...
I20260619 02:42:06.587443 18850 ts_manager.cc:194] Registered new tserver with Master: ee0f5d3172914c5e87a2fa76d6faf9fc (127.5.241.65:45197)
I20260619 02:42:06.588655 18850 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.65@KRBTEST.COM'} at 127.5.241.65:58769
I20260619 02:42:06.774715 19182 heartbeater.cc:464] Registering TS with master...
I20260619 02:42:06.774878 19182 heartbeater.cc:511] Master 127.5.241.126:38663 requested a full tablet report, sending...
I20260619 02:42:06.775290 18850 ts_manager.cc:194] Registered new tserver with Master: 73552824d4554814928278b51d067fa5 (127.5.241.66:45087)
I20260619 02:42:06.775779 18850 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.66@KRBTEST.COM'} at 127.5.241.66:54737
Jun 19 02:42:06 dist-test-slave-zpbg krb5kdc[17854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836926, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for HTTP/127.5.241.84@KRBTEST.COM
I20260619 02:42:06.977890 19316 heartbeater.cc:464] Registering TS with master...
I20260619 02:42:06.978044 19316 heartbeater.cc:511] Master 127.5.241.126:38663 requested a full tablet report, sending...
I20260619 02:42:06.978297 18850 ts_manager.cc:194] Registered new tserver with Master: 1ae98af952b04b94a24e1bd94a4d8526 (127.5.241.67:40225)
I20260619 02:42:06.978824 18850 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.67@KRBTEST.COM'} at 127.5.241.67:37305
I20260619 02:42:06.979063 6085 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:42:06.988152 19338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:06.980013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56644 (local address 127.5.241.126:38663)
0619 02:42:06.980172 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:06.980174 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:06.980188 (+ 14us) server_negotiation.cc:407] Connection header received
0619 02:42:06.980264 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:06.980265 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:06.980277 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:06.980377 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:42:06.981047 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:06.981811 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:06.982897 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:06.983207 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:06.983273 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:06.984785 (+ 1512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:42:06.984786 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:42:06.984787 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:42:06.984795 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:42:06.986553 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:06.986998 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:06.986999 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:06.987000 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:06.987062 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:06.987255 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:06.987257 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:06.987257 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:06.987387 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:42:06.987482 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:06.987962 (+ 480us) server_negotiation.cc:299] Negotiation successful
0619 02:42:06.988048 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":57,"threads_started":1}
I20260619 02:42:06.990823 18850 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56644:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:42:06.993088 18850 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:42:07.007401 19347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:06.996188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59576 (local address 127.5.241.66:45087)
0619 02:42:06.996439 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:06.996444 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:07.003013 (+ 6569us) server_negotiation.cc:407] Connection header received
0619 02:42:07.003143 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:07.003146 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:07.003164 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:07.003218 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:42:07.003504 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:07.004553 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:07.006078 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:07.006877 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:07.006929 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:07.006992 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:07.007266 (+ 274us) server_negotiation.cc:299] Negotiation successful
0619 02:42:07.007309 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":68,"threads_started":1}
I20260619 02:42:07.007688 19344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:06.996151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.67:40225 (local address 127.0.0.1:57246)
0619 02:42:07.003014 (+ 6863us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:07.003024 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:07.003085 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:07.003381 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:07.003382 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:07.003384 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:42:07.003428 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:07.003430 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:07.004919 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:07.004920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:07.006083 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:07.006086 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:07.007427 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:07.007428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:07.007491 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:07.007495 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:42:07.007543 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 02:42:07.007606 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6842,"thread_start_us":54,"threads_started":1}
I20260619 02:42:07.007887 19346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:06.995933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.66:45087 (local address 127.0.0.1:59576)
0619 02:42:07.002892 (+ 6959us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:07.002937 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:07.003040 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:07.003345 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:07.003348 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:07.003351 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:42:07.003424 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:07.003426 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:07.004658 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:07.004659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:07.005967 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:07.005970 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:07.007057 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:07.007058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:07.007139 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:07.007153 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 02:42:07.007745 (+ 592us) client_negotiation.cc:240] Negotiation successful
0619 02:42:07.007793 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6945,"thread_start_us":87,"threads_started":1}
I20260619 02:42:07.008225 19348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:06.996425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57246 (local address 127.5.241.67:40225)
0619 02:42:06.996605 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:06.996609 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:07.003151 (+ 6542us) server_negotiation.cc:407] Connection header received
0619 02:42:07.003193 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:07.003196 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:07.003212 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:07.003287 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:42:07.003498 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:07.004820 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:07.006182 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:07.006970 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:07.007038 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:07.007092 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:07.008093 (+ 1001us) server_negotiation.cc:299] Negotiation successful
0619 02:42:07.008138 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":62,"threads_started":1}
I20260619 02:42:07.008839 19116 tablet_service.cc:1467] Processing CreateTablet for tablet 1b12da6b074a486a9ae38afba4d4549f (DEFAULT_TABLE table=test-table [id=b74fd824199a4adf9997e16aba0ad276]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:42:07.009274 19116 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b12da6b074a486a9ae38afba4d4549f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:42:07.012372 19343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:06.995930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:45197 (local address 127.0.0.1:51086)
0619 02:42:07.002882 (+ 6952us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:07.002937 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:07.003017 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:07.008424 (+ 5407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:07.008426 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:07.008428 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:42:07.008476 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:07.008477 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:07.009629 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:07.009630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:07.010641 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:07.010644 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:07.012119 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:07.012120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:07.012183 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:07.012187 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:42:07.012236 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 02:42:07.012291 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6910,"thread_start_us":92,"threads_started":1}
I20260619 02:42:07.012640 19345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:06.996017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51086 (local address 127.5.241.65:45197)
0619 02:42:07.002886 (+ 6869us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:07.002890 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:07.008205 (+ 5315us) server_negotiation.cc:407] Connection header received
0619 02:42:07.008253 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:07.008255 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:07.008276 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:07.008566 (+ 290us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:42:07.008659 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:07.009543 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:07.010745 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:07.011952 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:07.012029 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:07.012084 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:07.012519 (+ 435us) server_negotiation.cc:299] Negotiation successful
0619 02:42:07.012565 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6823,"thread_start_us":39,"threads_started":1}
I20260619 02:42:07.013414 18982 tablet_service.cc:1467] Processing CreateTablet for tablet 1b12da6b074a486a9ae38afba4d4549f (DEFAULT_TABLE table=test-table [id=b74fd824199a4adf9997e16aba0ad276]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:42:07.013732 18982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b12da6b074a486a9ae38afba4d4549f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:42:07.013911 19250 tablet_service.cc:1467] Processing CreateTablet for tablet 1b12da6b074a486a9ae38afba4d4549f (DEFAULT_TABLE table=test-table [id=b74fd824199a4adf9997e16aba0ad276]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:42:07.014237 19250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b12da6b074a486a9ae38afba4d4549f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:42:07.018251 19349 tablet_bootstrap.cc:492] T 1b12da6b074a486a9ae38afba4d4549f P 73552824d4554814928278b51d067fa5: Bootstrap starting.
I20260619 02:42:07.018460 19351 tablet_bootstrap.cc:492] T 1b12da6b074a486a9ae38afba4d4549f P 1ae98af952b04b94a24e1bd94a4d8526: Bootstrap starting.
I20260619 02:42:07.019161 19351 tablet_bootstrap.cc:654] T 1b12da6b074a486a9ae38afba4d4549f P 1ae98af952b04b94a24e1bd94a4d8526: Neither blocks nor log segments found. Creating new log.
I20260619 02:42:07.019821 19349 tablet_bootstrap.cc:654] T 1b12da6b074a486a9ae38afba4d4549f P 73552824d4554814928278b51d067fa5: Neither blocks nor log segments found. Creating new log.
I20260619 02:42:07.020184 19349 log.cc:824] T 1b12da6b074a486a9ae38afba4d4549f P 73552824d4554814928278b51d067fa5: Log is configured to *not* fsync() on all Append() calls
I20260619 02:42:07.020571 19351 log.cc:824] T 1b12da6b074a486a9ae38afba4d4549f P 1ae98af952b04b94a24e1bd94a4d8526: Log is configured to *not* fsync() on all Append() calls
I20260619 02:42:07.021121 19350 tablet_bootstrap.cc:492] T 1b12da6b074a486a9ae38afba4d4549f P ee0f5d3172914c5e87a2fa76d6faf9fc: Bootstrap starting.
I20260619 02:42:07.021737 19350 tablet_bootstrap.cc:654] T 1b12da6b074a486a9ae38afba4d4549f P ee0f5d3172914c5e87a2fa76d6faf9fc: Neither blocks nor log segments found. Creating new log.
I20260619 02:42:07.022092 19350 log.cc:824] T 1b12da6b074a486a9ae38afba4d4549f P ee0f5d3172914c5e87a2fa76d6faf9fc: Log is configured to *not* fsync() on all Append() calls
I20260619 02:42:07.023058 19350 tablet_bootstrap.cc:492] T 1b12da6b074a486a9ae38afba4d4549f P ee0f5d3172914c5e87a2fa76d6faf9fc: No bootstrap required, opened a new log
I20260619 02:42:07.023056 19351 tablet_bootstrap.cc:492] T 1b12da6b074a486a9ae38afba4d4549f P 1ae98af952b04b94a24e1bd94a4d8526: No bootstrap required, opened a new log
I20260619 02:42:07.023128 19351 ts_tablet_manager.cc:1397] T 1b12da6b074a486a9ae38afba4d4549f P 1ae98af952b04b94a24e1bd94a4d8526: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260619 02:42:07.023238 19350 ts_tablet_manager.cc:1397] T 1b12da6b074a486a9ae38afba4d4549f P ee0f5d3172914c5e87a2fa76d6faf9fc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:42:07.026037 19351 raft_consensus.cc:348] T 1b12da6b074a486a9ae38afba4d4549f P 1ae98af952b04b94a24e1bd94a4d8526 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73552824d4554814928278b51d067fa5" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 45087 } } peers { permanent_uuid: "ee0f5d3172914c5e87a2fa76d6faf9fc" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 45197 } } peers { permanent_uuid: "1ae98af952b04b94a24e1bd94a4d8526" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 40225 } }
I20260619 02:42:07.026208 19349 tablet_bootstrap.cc:492] T 1b12da6b074a486a9ae38afba4d4549f P 73552824d4554814928278b51d067fa5: No bootstrap required, opened a new log
I20260619 02:42:07.026283 19349 ts_tablet_manager.cc:1397] T 1b12da6b074a486a9ae38afba4d4549f P 73552824d4554814928278b51d067fa5: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260619 02:42:07.026564 19351 raft_consensus.cc:374] T 1b12da6b074a486a9ae38afba4d4549f P 1ae98af952b04b94a24e1bd94a4d8526 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:42:07.026625 19351 raft_consensus.cc:729] T 1b12da6b074a486a9ae38afba4d4549f P 1ae98af952b04b94a24e1bd94a4d8526 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ae98af952b04b94a24e1bd94a4d8526, State: Initialized, Role: FOLLOWER
I20260619 02:42:07.026790 19351 consensus_queue.cc:260] T 1b12da6b074a486a9ae38afba4d4549f P 1ae98af952b04b94a24e1bd94a4d8526 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73552824d4554814928278b51d067fa5" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 45087 } } peers { permanent_uuid: "ee0f5d3172914c5e87a2fa76d6faf9fc" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 45197 } } peers { permanent_uuid: "1ae98af952b04b94a24e1bd94a4d8526" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 40225 } }
I20260619 02:42:07.027091 19351 ts_tablet_manager.cc:1428] T 1b12da6b074a486a9ae38afba4d4549f P 1ae98af952b04b94a24e1bd94a4d8526: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
I20260619 02:42:07.027207 19316 heartbeater.cc:503] Master 127.5.241.126:38663 was elected leader, sending a full tablet report...
I20260619 02:42:07.028482 19349 raft_consensus.cc:348] T 1b12da6b074a486a9ae38afba4d4549f P 73552824d4554814928278b51d067fa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73552824d4554814928278b51d067fa5" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 45087 } } peers { permanent_uuid: "ee0f5d3172914c5e87a2fa76d6faf9fc" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 45197 } } peers { permanent_uuid: "1ae98af952b04b94a24e1bd94a4d8526" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 40225 } }
I20260619 02:42:07.028638 19349 raft_consensus.cc:374] T 1b12da6b074a486a9ae38afba4d4549f P 73552824d4554814928278b51d067fa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:42:07.028671 19349 raft_consensus.cc:729] T 1b12da6b074a486a9ae38afba4d4549f P 73552824d4554814928278b51d067fa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73552824d4554814928278b51d067fa5, State: Initialized, Role: FOLLOWER
I20260619 02:42:07.028805 19349 consensus_queue.cc:260] T 1b12da6b074a486a9ae38afba4d4549f P 73552824d4554814928278b51d067fa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73552824d4554814928278b51d067fa5" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 45087 } } peers { permanent_uuid: "ee0f5d3172914c5e87a2fa76d6faf9fc" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 45197 } } peers { permanent_uuid: "1ae98af952b04b94a24e1bd94a4d8526" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 40225 } }
I20260619 02:42:07.029058 19349 ts_tablet_manager.cc:1428] T 1b12da6b074a486a9ae38afba4d4549f P 73552824d4554814928278b51d067fa5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 02:42:07.029207 19182 heartbeater.cc:503] Master 127.5.241.126:38663 was elected leader, sending a full tablet report...
I20260619 02:42:07.029839 19350 raft_consensus.cc:348] T 1b12da6b074a486a9ae38afba4d4549f P ee0f5d3172914c5e87a2fa76d6faf9fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73552824d4554814928278b51d067fa5" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 45087 } } peers { permanent_uuid: "ee0f5d3172914c5e87a2fa76d6faf9fc" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 45197 } } peers { permanent_uuid: "1ae98af952b04b94a24e1bd94a4d8526" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 40225 } }
I20260619 02:42:07.030011 19350 raft_consensus.cc:374] T 1b12da6b074a486a9ae38afba4d4549f P ee0f5d3172914c5e87a2fa76d6faf9fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:42:07.030050 19350 raft_consensus.cc:729] T 1b12da6b074a486a9ae38afba4d4549f P ee0f5d3172914c5e87a2fa76d6faf9fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee0f5d3172914c5e87a2fa76d6faf9fc, State: Initialized, Role: FOLLOWER
I20260619 02:42:07.030210 19350 consensus_queue.cc:260] T 1b12da6b074a486a9ae38afba4d4549f P ee0f5d3172914c5e87a2fa76d6faf9fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73552824d4554814928278b51d067fa5" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 45087 } } peers { permanent_uuid: "ee0f5d3172914c5e87a2fa76d6faf9fc" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 45197 } } peers { permanent_uuid: "1ae98af952b04b94a24e1bd94a4d8526" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 40225 } }
I20260619 02:42:07.030472 19350 ts_tablet_manager.cc:1428] T 1b12da6b074a486a9ae38afba4d4549f P ee0f5d3172914c5e87a2fa76d6faf9fc: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
I20260619 02:42:07.030588 19048 heartbeater.cc:503] Master 127.5.241.126:38663 was elected leader, sending a full tablet report...
W20260619 02:42:07.065856 19049 tablet.cc:2369] T 1b12da6b074a486a9ae38afba4d4549f P ee0f5d3172914c5e87a2fa76d6faf9fc: Can't schedule compaction. Clean time has not been advanced past its initial value.
Jun 19 02:42:07 dist-test-slave-zpbg krb5kdc[17854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836926, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for HTTP/127.5.241.84@KRBTEST.COM
I20260619 02:42:07.152204 19355 raft_consensus.cc:482] T 1b12da6b074a486a9ae38afba4d4549f P 1ae98af952b04b94a24e1bd94a4d8526 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:42:07.152355 19355 raft_consensus.cc:504] T 1b12da6b074a486a9ae38afba4d4549f P 1ae98af952b04b94a24e1bd94a4d8526 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73552824d4554814928278b51d067fa5" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 45087 } } peers { permanent_uuid: "ee0f5d3172914c5e87a2fa76d6faf9fc" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 45197 } } peers { permanent_uuid: "1ae98af952b04b94a24e1bd94a4d8526" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 40225 } }
I20260619 02:42:07.152778 19355 leader_election.cc:288] T 1b12da6b074a486a9ae38afba4d4549f P 1ae98af952b04b94a24e1bd94a4d8526 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73552824d4554814928278b51d067fa5 (127.5.241.66:45087), ee0f5d3172914c5e87a2fa76d6faf9fc (127.5.241.65:45197)
I20260619 02:42:07.158857 19347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:07.153637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:44959 (local address 127.5.241.66:45087)
0619 02:42:07.153773 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:07.153775 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:07.155147 (+ 1372us) server_negotiation.cc:407] Connection header received
0619 02:42:07.155281 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:07.155282 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:07.155294 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:07.155326 (+ 32us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:42:07.155559 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:07.156302 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:07.157816 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:07.158398 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:07.158452 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:07.158464 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:07.158710 (+ 246us) server_negotiation.cc:299] Negotiation successful
0619 02:42:07.158753 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260619 02:42:07.159375 19361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:07.153078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.66:45087 (local address 127.5.241.67:44959)
0619 02:42:07.155028 (+ 1950us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:07.155066 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:07.155192 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:07.155400 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:07.155401 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:07.155403 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:42:07.155481 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:07.155484 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:07.156414 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:07.156415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:07.157701 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:07.157705 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:07.158545 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:07.158546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:07.158618 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:07.158623 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 02:42:07.159222 (+ 599us) client_negotiation.cc:240] Negotiation successful
0619 02:42:07.159272 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1931,"thread_start_us":118,"threads_started":1}
I20260619 02:42:07.159876 19136 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b12da6b074a486a9ae38afba4d4549f" candidate_uuid: "1ae98af952b04b94a24e1bd94a4d8526" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73552824d4554814928278b51d067fa5" is_pre_election: true
I20260619 02:42:07.160058 19136 raft_consensus.cc:2454] T 1b12da6b074a486a9ae38afba4d4549f P 73552824d4554814928278b51d067fa5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ae98af952b04b94a24e1bd94a4d8526 in term 0.
I20260619 02:42:07.160423 19203 leader_election.cc:302] T 1b12da6b074a486a9ae38afba4d4549f P 1ae98af952b04b94a24e1bd94a4d8526 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ae98af952b04b94a24e1bd94a4d8526, 73552824d4554814928278b51d067fa5; no voters:
I20260619 02:42:07.160567 19355 raft_consensus.cc:2781] T 1b12da6b074a486a9ae38afba4d4549f P 1ae98af952b04b94a24e1bd94a4d8526 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:42:07.160610 19355 raft_consensus.cc:482] T 1b12da6b074a486a9ae38afba4d4549f P 1ae98af952b04b94a24e1bd94a4d8526 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:42:07.160631 19355 raft_consensus.cc:3037] T 1b12da6b074a486a9ae38afba4d4549f P 1ae98af952b04b94a24e1bd94a4d8526 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:42:07.161577 19355 raft_consensus.cc:504] T 1b12da6b074a486a9ae38afba4d4549f P 1ae98af952b04b94a24e1bd94a4d8526 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73552824d4554814928278b51d067fa5" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 45087 } } peers { permanent_uuid: "ee0f5d3172914c5e87a2fa76d6faf9fc" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 45197 } } peers { permanent_uuid: "1ae98af952b04b94a24e1bd94a4d8526" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 40225 } }
I20260619 02:42:07.161720 19355 leader_election.cc:288] T 1b12da6b074a486a9ae38afba4d4549f P 1ae98af952b04b94a24e1bd94a4d8526 [CANDIDATE]: Term 1 election: Requested vote from peers 73552824d4554814928278b51d067fa5 (127.5.241.66:45087), ee0f5d3172914c5e87a2fa76d6faf9fc (127.5.241.65:45197)
I20260619 02:42:07.162216 19136 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b12da6b074a486a9ae38afba4d4549f" candidate_uuid: "1ae98af952b04b94a24e1bd94a4d8526" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73552824d4554814928278b51d067fa5"
I20260619 02:42:07.162299 19136 raft_consensus.cc:3037] T 1b12da6b074a486a9ae38afba4d4549f P 73552824d4554814928278b51d067fa5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:42:07.163213 19136 raft_consensus.cc:2454] T 1b12da6b074a486a9ae38afba4d4549f P 73552824d4554814928278b51d067fa5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ae98af952b04b94a24e1bd94a4d8526 in term 1.
I20260619 02:42:07.163443 19203 leader_election.cc:302] T 1b12da6b074a486a9ae38afba4d4549f P 1ae98af952b04b94a24e1bd94a4d8526 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ae98af952b04b94a24e1bd94a4d8526, 73552824d4554814928278b51d067fa5; no voters:
I20260619 02:42:07.163563 19355 raft_consensus.cc:2781] T 1b12da6b074a486a9ae38afba4d4549f P 1ae98af952b04b94a24e1bd94a4d8526 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:42:07.163702 19355 raft_consensus.cc:686] T 1b12da6b074a486a9ae38afba4d4549f P 1ae98af952b04b94a24e1bd94a4d8526 [term 1 LEADER]: Becoming Leader. State: Replica: 1ae98af952b04b94a24e1bd94a4d8526, State: Running, Role: LEADER
I20260619 02:42:07.163789 19355 consensus_queue.cc:237] T 1b12da6b074a486a9ae38afba4d4549f P 1ae98af952b04b94a24e1bd94a4d8526 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73552824d4554814928278b51d067fa5" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 45087 } } peers { permanent_uuid: "ee0f5d3172914c5e87a2fa76d6faf9fc" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 45197 } } peers { permanent_uuid: "1ae98af952b04b94a24e1bd94a4d8526" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 40225 } }
I20260619 02:42:07.164631 18849 catalog_manager.cc:5495] T 1b12da6b074a486a9ae38afba4d4549f P 1ae98af952b04b94a24e1bd94a4d8526 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ae98af952b04b94a24e1bd94a4d8526 (127.5.241.67). New cstate: current_term: 1 leader_uuid: "1ae98af952b04b94a24e1bd94a4d8526" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73552824d4554814928278b51d067fa5" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 45087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee0f5d3172914c5e87a2fa76d6faf9fc" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 45197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ae98af952b04b94a24e1bd94a4d8526" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 40225 } health_report { overall_health: HEALTHY } } }
I20260619 02:42:07.166510 19345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:07.153703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:49225 (local address 127.5.241.65:45197)
0619 02:42:07.153766 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:07.153768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:07.162930 (+ 9162us) server_negotiation.cc:407] Connection header received
0619 02:42:07.163040 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:07.163041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:07.163052 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:07.163089 (+ 37us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:42:07.163306 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:07.164026 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:07.165546 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:07.166102 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:07.166154 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:07.166162 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:07.166398 (+ 236us) server_negotiation.cc:299] Negotiation successful
0619 02:42:07.166439 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 02:42:07.166901 19360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:07.153314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:45197 (local address 127.5.241.67:49225)
0619 02:42:07.162847 (+ 9533us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:07.162860 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:07.162962 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:07.163156 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:07.163157 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:07.163158 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:42:07.163218 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:07.163220 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:07.164132 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:07.164132 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:07.165451 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:07.165454 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:07.166224 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:07.166224 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:07.166315 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:07.166319 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:42:07.166751 (+ 432us) client_negotiation.cc:240] Negotiation successful
0619 02:42:07.166818 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9512,"thread_start_us":59,"threads_started":1}
I20260619 02:42:07.167368 19002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b12da6b074a486a9ae38afba4d4549f" candidate_uuid: "1ae98af952b04b94a24e1bd94a4d8526" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee0f5d3172914c5e87a2fa76d6faf9fc" is_pre_election: true
I20260619 02:42:07.167531 19002 raft_consensus.cc:2454] T 1b12da6b074a486a9ae38afba4d4549f P ee0f5d3172914c5e87a2fa76d6faf9fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ae98af952b04b94a24e1bd94a4d8526 in term 0.
I20260619 02:42:07.167670 19001 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b12da6b074a486a9ae38afba4d4549f" candidate_uuid: "1ae98af952b04b94a24e1bd94a4d8526" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee0f5d3172914c5e87a2fa76d6faf9fc"
I20260619 02:42:07.167740 19001 raft_consensus.cc:3037] T 1b12da6b074a486a9ae38afba4d4549f P ee0f5d3172914c5e87a2fa76d6faf9fc [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:42:07.168574 19001 raft_consensus.cc:2454] T 1b12da6b074a486a9ae38afba4d4549f P ee0f5d3172914c5e87a2fa76d6faf9fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ae98af952b04b94a24e1bd94a4d8526 in term 1.
I20260619 02:42:07.184358 19348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:07.181029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57258 (local address 127.5.241.67:40225)
0619 02:42:07.181069 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:07.181071 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:07.181078 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:42:07.181116 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:07.181117 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:07.181136 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:07.181193 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:42:07.181366 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:07.182121 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:07.183271 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:07.183575 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:07.183639 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:07.183852 (+ 213us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:42:07.183978 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:42:07.184045 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:07.184259 (+ 214us) server_negotiation.cc:299] Negotiation successful
0619 02:42:07.184279 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 02:42:07.186606 19001 raft_consensus.cc:1261] T 1b12da6b074a486a9ae38afba4d4549f P ee0f5d3172914c5e87a2fa76d6faf9fc [term 1 FOLLOWER]: Refusing update from remote peer 1ae98af952b04b94a24e1bd94a4d8526: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:42:07.186606 19136 raft_consensus.cc:1261] T 1b12da6b074a486a9ae38afba4d4549f P 73552824d4554814928278b51d067fa5 [term 1 FOLLOWER]: Refusing update from remote peer 1ae98af952b04b94a24e1bd94a4d8526: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:42:07.186961 19355 consensus_queue.cc:1035] T 1b12da6b074a486a9ae38afba4d4549f P 1ae98af952b04b94a24e1bd94a4d8526 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73552824d4554814928278b51d067fa5" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 45087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:42:07.187071 19362 consensus_queue.cc:1035] T 1b12da6b074a486a9ae38afba4d4549f P 1ae98af952b04b94a24e1bd94a4d8526 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee0f5d3172914c5e87a2fa76d6faf9fc" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 45197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:42:07.191306 19367 mvcc.cc:205] Tried to move back new op lower bound from 7298404053753176064 to 7298404053663416320. Current Snapshot: MvccSnapshot[applied={T|T < 7298404053753176064}]
I20260619 02:42:07.194526 18850 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56644:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:42:07.194623 18850 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56644:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:42:07.195489 18850 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bc6a5cfb7f254f74a8a1e7c431318d95: Sending DeleteTablet for 3 replicas of tablet 1b12da6b074a486a9ae38afba4d4549f
I20260619 02:42:07.195940 19116 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b12da6b074a486a9ae38afba4d4549f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:42:07 UTC) from {username='kudu', principal='kudu/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:59576
I20260619 02:42:07.196144 18982 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b12da6b074a486a9ae38afba4d4549f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:42:07 UTC) from {username='kudu', principal='kudu/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:51086
I20260619 02:42:07.196619 19250 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b12da6b074a486a9ae38afba4d4549f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:42:07 UTC) from {username='kudu', principal='kudu/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:57246
I20260619 02:42:07.197130 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 18888
I20260619 02:42:07.200419 19373 tablet_replica.cc:330] T 1b12da6b074a486a9ae38afba4d4549f P 73552824d4554814928278b51d067fa5: stopping tablet replica
I20260619 02:42:07.200567 19373 raft_consensus.cc:2229] T 1b12da6b074a486a9ae38afba4d4549f P 73552824d4554814928278b51d067fa5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:42:07.200708 19373 raft_consensus.cc:2258] T 1b12da6b074a486a9ae38afba4d4549f P 73552824d4554814928278b51d067fa5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:42:07.200877 19375 tablet_replica.cc:330] T 1b12da6b074a486a9ae38afba4d4549f P 1ae98af952b04b94a24e1bd94a4d8526: stopping tablet replica
I20260619 02:42:07.200978 19375 raft_consensus.cc:2229] T 1b12da6b074a486a9ae38afba4d4549f P 1ae98af952b04b94a24e1bd94a4d8526 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:42:07.201095 19375 raft_consensus.cc:2258] T 1b12da6b074a486a9ae38afba4d4549f P 1ae98af952b04b94a24e1bd94a4d8526 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:42:07.201402 19373 ts_tablet_manager.cc:1905] T 1b12da6b074a486a9ae38afba4d4549f P 73552824d4554814928278b51d067fa5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:42:07.203577 19373 ts_tablet_manager.cc:1918] T 1b12da6b074a486a9ae38afba4d4549f P 73552824d4554814928278b51d067fa5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:42:07.203712 19373 log.cc:1196] T 1b12da6b074a486a9ae38afba4d4549f P 73552824d4554814928278b51d067fa5: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-1/wal/wals/1b12da6b074a486a9ae38afba4d4549f
W20260619 02:42:07.204176 19205 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.241.65:45197 (error 108)
I20260619 02:42:07.204380 19373 ts_tablet_manager.cc:1939] T 1b12da6b074a486a9ae38afba4d4549f P 73552824d4554814928278b51d067fa5: Deleting consensus metadata
I20260619 02:42:07.205003 18834 catalog_manager.cc:4841] TS 73552824d4554814928278b51d067fa5 (127.5.241.66:45087): tablet 1b12da6b074a486a9ae38afba4d4549f (table test-table [id=b74fd824199a4adf9997e16aba0ad276]) successfully deleted
I20260619 02:42:07.207086 19375 ts_tablet_manager.cc:1905] T 1b12da6b074a486a9ae38afba4d4549f P 1ae98af952b04b94a24e1bd94a4d8526: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:42:07.208950 19375 ts_tablet_manager.cc:1918] T 1b12da6b074a486a9ae38afba4d4549f P 1ae98af952b04b94a24e1bd94a4d8526: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:42:07.209088 19375 log.cc:1196] T 1b12da6b074a486a9ae38afba4d4549f P 1ae98af952b04b94a24e1bd94a4d8526: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836772852275-6085-0/minicluster-data/ts-2/wal/wals/1b12da6b074a486a9ae38afba4d4549f
I20260619 02:42:07.209497 19375 ts_tablet_manager.cc:1939] T 1b12da6b074a486a9ae38afba4d4549f P 1ae98af952b04b94a24e1bd94a4d8526: Deleting consensus metadata
I20260619 02:42:07.209969 18836 catalog_manager.cc:4841] TS 1ae98af952b04b94a24e1bd94a4d8526 (127.5.241.67:40225): tablet 1b12da6b074a486a9ae38afba4d4549f (table test-table [id=b74fd824199a4adf9997e16aba0ad276]) successfully deleted
W20260619 02:42:07.210321 18836 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.241.65:45197 (error 108)
I20260619 02:42:07.211171 19360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:07.211057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:45197 (local address 127.5.241.67:38425)
0619 02:42:07.211099 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:07.211134 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.65:45197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260619 02:42:07.211196 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 19052
I20260619 02:42:07.218089 19346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:07.217967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:45197 (local address 127.0.0.1:51102)
0619 02:42:07.218014 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:07.218048 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.65:45197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260619 02:42:07.218335 18836 catalog_manager.cc:4584] TS ee0f5d3172914c5e87a2fa76d6faf9fc (127.5.241.65:45197): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1b12da6b074a486a9ae38afba4d4549f: Network error: Client connection negotiation failed: client connection to 127.5.241.65:45197: connect: Connection refused (error 111)
I20260619 02:42:07.218583 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 19185
I20260619 02:42:07.224483 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 18817
I20260619 02:42:07.231184 6085 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260619 02:42:07.575493 6085 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260619 02:42:07.575587 6085 mini_ranger.cc:66] Stopping Ranger...
I20260619 02:42:07.919752 6085 mini_ranger.cc:68] Stopped Ranger
2026-06-19 02:42:07.919 UTC [17884] LOG: received smart shutdown request
2026-06-19 02:42:07.923 UTC [17884] LOG: background worker "logical replication launcher" (PID 17893) exited with exit code 1
2026-06-19 02:42:07.924 UTC [17888] LOG: shutting down
2026-06-19 02:42:08.101 UTC [17884] LOG: database system is shut down
2026-06-19T02:42:08Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (48582 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:42:08 dist-test-slave-zpbg krb5kdc[19386](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:42:08 dist-test-slave-zpbg krb5kdc[19386](info): set up 2 sockets
Jun 19 02:42:08 dist-test-slave-zpbg krb5kdc[19386](info): commencing operation
krb5kdc: starting...
W20260619 02:42:09.246734 6085 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Jun 19 02:42:09 dist-test-slave-zpbg krb5kdc[19386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836929, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:42:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:42:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.241.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
I20260619 02:42:09.365469 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:33863
--webserver_interface=127.5.241.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:38957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:33863,127.5.241.125:43445,127.5.241.124:35771
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:42:09.446420 19399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:42:09.446642 19399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:42:09.446664 19399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:42:09.447788 19399 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:42:09.447826 19399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:42:09.447841 19399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:42:09.447854 19399 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:42:09.447867 19399 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:42:09.449328 19399 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:38957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:33863,127.5.241.125:43445,127.5.241.124:35771
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:33863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.241.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19399
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:42:09.449525 19399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:42:09.449694 19399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:42:09.454672 19405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:09.454672 19407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:09.454802 19404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:42:09.455052 19399 server_base.cc:1034] running on GCE node
I20260619 02:42:09.455334 19399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:42:09.455626 19399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:42:09.456796 19399 hybrid_clock.cc:648] HybridClock initialized: now 1781836929456773 us; error 42 us; skew 500 ppm
Jun 19 02:42:09 dist-test-slave-zpbg krb5kdc[19386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836929, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:42:09.464326 19399 init.cc:377] Logged in from keytab as kudu/127.5.241.126@KRBTEST.COM (short username kudu)
I20260619 02:42:09.464825 19399 webserver.cc:460] Webserver started at http://127.5.241.126:46377/ using document root <none> and password file <none>
I20260619 02:42:09.465018 19399 fs_manager.cc:359] Metadata directory not provided
I20260619 02:42:09.465061 19399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:42:09.465154 19399 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:42:09.466190 19399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/minicluster-data/master-0/data/instance:
uuid: "bd6fe680bd2441dfb41352c041263edc"
format_stamp: "Formatted at 2026-06-19 02:42:09 on dist-test-slave-zpbg"
server_key: "4c41d696ba861642a8f324be7a3f3f98"
server_key_iv: "f9596af28f7914c6026e2ce83959dff1"
server_key_version: "encryptionkey@0"
I20260619 02:42:09.466511 19399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/minicluster-data/master-0/wal/instance:
uuid: "bd6fe680bd2441dfb41352c041263edc"
format_stamp: "Formatted at 2026-06-19 02:42:09 on dist-test-slave-zpbg"
server_key: "4c41d696ba861642a8f324be7a3f3f98"
server_key_iv: "f9596af28f7914c6026e2ce83959dff1"
server_key_version: "encryptionkey@0"
I20260619 02:42:09.467940 19399 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:42:09.468652 19414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:09.468866 19399 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:42:09.468959 19399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "bd6fe680bd2441dfb41352c041263edc"
format_stamp: "Formatted at 2026-06-19 02:42:09 on dist-test-slave-zpbg"
server_key: "4c41d696ba861642a8f324be7a3f3f98"
server_key_iv: "f9596af28f7914c6026e2ce83959dff1"
server_key_version: "encryptionkey@0"
I20260619 02:42:09.469018 19399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:42:09.481881 19399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:42:09.484114 19399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:42:09.484231 19399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:42:09.488065 19399 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.126:33863
I20260619 02:42:09.488094 19466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.126:33863 every 8 connection(s)
I20260619 02:42:09.488457 19399 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:42:09.489362 19467 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:42:09.490553 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 19399
I20260619 02:42:09.490671 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/minicluster-data/master-0/wal/instance
I20260619 02:42:09.490912 6085 external_mini_cluster.cc:1428] Setting key 666bfcbc90ac3c6882d90e94501515b2
I20260619 02:42:09.491196 19467 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.241.126" port: 33863 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:42:09 dist-test-slave-zpbg krb5kdc[19386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836929, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.241.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.125@KRBTEST.COM" created.
I20260619 02:42:09.501513 19469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:09.491991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:33863 (local address 127.0.0.1:56762)
0619 02:42:09.492176 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:09.492222 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:09.492787 (+ 565us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:09.493101 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:09.493104 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:09.493388 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:42:09.493542 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:09.493546 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:09.494565 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:09.494566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:09.495304 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:09.495306 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:09.495830 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:09.495830 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:09.495895 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:09.496512 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:42:09.496521 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:42:09.497922 (+ 1401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:42:09.499903 (+ 1981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:09.499910 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:09.499912 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:09.500164 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:09.500441 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:09.500442 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:09.500443 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:09.500534 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:09.500902 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:42:09.500906 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:42:09.500967 (+ 61us) client_negotiation.cc:769] Sending connection context
0619 02:42:09.501080 (+ 113us) client_negotiation.cc:240] Negotiation successful
0619 02:42:09.501316 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":93,"thread_start_us":89,"threads_started":1}
I20260619 02:42:09.501511 19471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:09.492166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56762 (local address 127.5.241.126:33863)
0619 02:42:09.492503 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:09.492508 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:09.492547 (+ 39us) server_negotiation.cc:407] Connection header received
0619 02:42:09.492956 (+ 409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:09.492958 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:09.492977 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:09.493093 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:42:09.493652 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:09.494452 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:09.495404 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:09.495725 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:09.495859 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:09.498025 (+ 2166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:42:09.498031 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:42:09.498032 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:42:09.498039 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:42:09.499793 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:09.500257 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:09.500260 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:09.500260 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:09.500385 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:09.500602 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:09.500603 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:09.500603 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:09.500830 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:42:09.500924 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:09.501110 (+ 186us) server_negotiation.cc:299] Negotiation successful
0619 02:42:09.501252 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":94,"threads_started":1}
I20260619 02:42:09.502583 19467 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.241.125" port: 43445 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:42:09.503022 19469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:09.502898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.125:43445 (local address 127.0.0.1:36902)
0619 02:42:09.502958 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:09.503001 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.125:43445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260619 02:42:09.503189 19418 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.241.125:43445: connect: Connection refused (error 111)
I20260619 02:42:09.503686 19469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:09.503577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.125:43445 (local address 127.0.0.1:36914)
0619 02:42:09.503632 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:09.503664 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.125:43445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260619 02:42:09.503840 19467 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.241.125:43445: Network error: Client connection negotiation failed: client connection to 127.5.241.125:43445: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.125.keytab.
Entry for principal kudu/127.5.241.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.125.keytab.
I20260619 02:42:09.517577 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.125:43445
--webserver_interface=127.5.241.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.125.keytab
--principal=kudu/127.5.241.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:38957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:33863,127.5.241.125:43445,127.5.241.124:35771
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:42:09.553123 19467 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.241.125" port: 43445 } attempt: 1
I20260619 02:42:09.553776 19469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:09.553600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.125:43445 (local address 127.0.0.1:36926)
0619 02:42:09.553696 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:09.553742 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.125:43445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260619 02:42:09.554242 19469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:09.554144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.125:43445 (local address 127.0.0.1:36932)
0619 02:42:09.554196 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:09.554223 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.125:43445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260619 02:42:09.554358 19467 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.241.125:43445: Network error: Client connection negotiation failed: client connection to 127.5.241.125:43445: connect: Connection refused (error 111)
W20260619 02:42:09.599488 19474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:42:09.599758 19474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:42:09.599802 19474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:42:09.600980 19474 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:42:09.601058 19474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:42:09.601085 19474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:42:09.601118 19474 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:42:09.601141 19474 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:42:09.602829 19474 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:38957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:33863,127.5.241.125:43445,127.5.241.124:35771
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.125.keytab
--principal=kudu/127.5.241.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.125:43445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.241.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19474
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:42:09.603117 19474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:42:09.603358 19474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:42:09.608374 19480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:09.608395 19479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:09.608394 19482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:42:09.608678 19474 server_base.cc:1034] running on GCE node
I20260619 02:42:09.608948 19474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:42:09.609293 19474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:42:09.610464 19474 hybrid_clock.cc:648] HybridClock initialized: now 1781836929610465 us; error 44 us; skew 500 ppm
Jun 19 02:42:09 dist-test-slave-zpbg krb5kdc[19386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836929, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:42:09.613047 19474 init.cc:377] Logged in from keytab as kudu/127.5.241.125@KRBTEST.COM (short username kudu)
I20260619 02:42:09.613535 19474 webserver.cc:460] Webserver started at http://127.5.241.125:38093/ using document root <none> and password file <none>
I20260619 02:42:09.613770 19474 fs_manager.cc:359] Metadata directory not provided
I20260619 02:42:09.613843 19474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:42:09.614005 19474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:42:09.615249 19474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/minicluster-data/master-1/data/instance:
uuid: "15f29de8c8094db1a3745c20e37ef503"
format_stamp: "Formatted at 2026-06-19 02:42:09 on dist-test-slave-zpbg"
server_key: "19e1af32bb73702590d40309ade8f868"
server_key_iv: "673d2c883cceb87a7506a3339e9a1389"
server_key_version: "encryptionkey@0"
I20260619 02:42:09.615612 19474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/minicluster-data/master-1/wal/instance:
uuid: "15f29de8c8094db1a3745c20e37ef503"
format_stamp: "Formatted at 2026-06-19 02:42:09 on dist-test-slave-zpbg"
server_key: "19e1af32bb73702590d40309ade8f868"
server_key_iv: "673d2c883cceb87a7506a3339e9a1389"
server_key_version: "encryptionkey@0"
I20260619 02:42:09.616904 19474 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:42:09.617609 19489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:09.617770 19474 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:42:09.617878 19474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/minicluster-data/master-1/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/minicluster-data/master-1/wal
uuid: "15f29de8c8094db1a3745c20e37ef503"
format_stamp: "Formatted at 2026-06-19 02:42:09 on dist-test-slave-zpbg"
server_key: "19e1af32bb73702590d40309ade8f868"
server_key_iv: "673d2c883cceb87a7506a3339e9a1389"
server_key_version: "encryptionkey@0"
I20260619 02:42:09.617950 19474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:42:09.622525 19467 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.241.125" port: 43445 } attempt: 2
I20260619 02:42:09.622987 19469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:09.622850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.125:43445 (local address 127.0.0.1:36944)
0619 02:42:09.622911 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:09.622952 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.125:43445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260619 02:42:09.623370 19469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:09.623239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.125:43445 (local address 127.0.0.1:36954)
0619 02:42:09.623302 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:09.623347 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.125:43445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260619 02:42:09.623512 19467 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.241.125:43445: Network error: Client connection negotiation failed: client connection to 127.5.241.125:43445: connect: Connection refused (error 111)
I20260619 02:42:09.639937 19474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:42:09.641857 19474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:42:09.641969 19474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:42:09.646203 19474 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.125:43445
I20260619 02:42:09.646301 19541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.125:43445 every 8 connection(s)
I20260619 02:42:09.646621 19474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/minicluster-data/master-1/data/info.pb
I20260619 02:42:09.647567 19542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:42:09.649246 19542 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.241.126" port: 33863 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:42:09.652616 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 19474
I20260619 02:42:09.652741 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/minicluster-data/master-1/wal/instance
I20260619 02:42:09.652926 6085 external_mini_cluster.cc:1428] Setting key 33cb851891595a0fbafe292387c2d242
Jun 19 02:42:09 dist-test-slave-zpbg krb5kdc[19386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836929, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.125@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:42:09.659211 19471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:09.650063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56776 (local address 127.5.241.126:33863)
0619 02:42:09.650138 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:09.650140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:09.650383 (+ 243us) server_negotiation.cc:407] Connection header received
0619 02:42:09.650935 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:09.650936 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:09.650951 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:09.651008 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:42:09.651659 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:09.652140 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:09.653328 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:09.653674 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:09.653756 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:09.656251 (+ 2495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:42:09.656253 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:42:09.656253 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:42:09.656267 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:42:09.657727 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:09.658168 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:09.658169 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:09.658169 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:09.658232 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:09.658501 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:09.658502 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:09.658503 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:09.658624 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:42:09.658698 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:09.659028 (+ 330us) server_negotiation.cc:299] Negotiation successful
0619 02:42:09.659116 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 02:42:09.659420 19544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:09.650026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:33863 (local address 127.0.0.1:56776)
0619 02:42:09.650255 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:09.650294 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:09.650748 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:09.651083 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:09.651088 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:09.651368 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:42:09.651560 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:09.651564 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:09.652244 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:09.652246 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:09.653227 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:09.653230 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:09.653788 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:09.653788 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:09.653897 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:09.654471 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:42:09.654486 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:42:09.656138 (+ 1652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:42:09.657833 (+ 1695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:09.657841 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:09.657843 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:09.658087 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:09.658303 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:09.658304 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:09.658305 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:09.658421 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:09.658703 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:42:09.658710 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:42:09.658892 (+ 182us) client_negotiation.cc:769] Sending connection context
0619 02:42:09.659008 (+ 116us) client_negotiation.cc:240] Negotiation successful
0619 02:42:09.659168 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":114,"threads_started":1}
I20260619 02:42:09.659917 19542 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.241.125" port: 43445 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.5.241.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.124@KRBTEST.COM" created.
Jun 19 02:42:09 dist-test-slave-zpbg krb5kdc[19386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836929, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.125@KRBTEST.COM for kudu/127.5.241.125@KRBTEST.COM
I20260619 02:42:09.667454 19544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:09.660235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.125:43445 (local address 127.0.0.1:36964)
0619 02:42:09.660279 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:09.660289 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:09.660346 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:09.660680 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:09.660681 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:09.660789 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:42:09.660838 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:09.660839 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:09.661704 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:09.661705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:09.662323 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:09.662325 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:09.662913 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:09.662914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:09.662985 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:09.663433 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:42:09.663438 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:42:09.664597 (+ 1159us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:42:09.666377 (+ 1780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:09.666378 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:09.666379 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:09.666539 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:09.666758 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:09.666759 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:09.666759 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:09.666832 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:09.667132 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:42:09.667133 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:42:09.667170 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 02:42:09.667242 (+ 72us) client_negotiation.cc:240] Negotiation successful
0619 02:42:09.667339 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260619 02:42:09.667527 19546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:09.660365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36964 (local address 127.5.241.125:43445)
0619 02:42:09.660532 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:09.660537 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:09.660553 (+ 16us) server_negotiation.cc:407] Connection header received
0619 02:42:09.660580 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:09.660581 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:09.660591 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:09.660650 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:42:09.660917 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:09.661603 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:09.662415 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:09.662823 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:09.662919 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:09.664697 (+ 1778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:42:09.664705 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:42:09.664705 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:42:09.664712 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:42:09.666252 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:09.666633 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:09.666636 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:09.666636 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:09.666692 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:09.666921 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:09.666922 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:09.666922 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:09.667061 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:42:09.667155 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:09.667271 (+ 116us) server_negotiation.cc:299] Negotiation successful
0619 02:42:09.667398 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":141,"thread_start_us":94,"threads_started":1}
I20260619 02:42:09.668107 19542 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.241.124" port: 35771 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:42:09.668481 19544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:09.668377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.124:35771 (local address 127.0.0.1:42638)
0619 02:42:09.668418 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:09.668460 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.124:35771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260619 02:42:09.668658 19490 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.241.124:35771: connect: Connection refused (error 111)
I20260619 02:42:09.669126 19544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:09.669041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.124:35771 (local address 127.0.0.1:42654)
0619 02:42:09.669086 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:09.669109 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.124:35771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260619 02:42:09.669250 19542 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.241.124:35771: Network error: Client connection negotiation failed: client connection to 127.5.241.124:35771: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.124.keytab.
Entry for principal kudu/127.5.241.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.124.keytab.
I20260619 02:42:09.678072 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.124:35771
--webserver_interface=127.5.241.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.124.keytab
--principal=kudu/127.5.241.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:38957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:33863,127.5.241.125:43445,127.5.241.124:35771
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:42:09.714720 19467 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.241.125" port: 43445 } attempt: 3
I20260619 02:42:09.718505 19542 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.241.124" port: 35771 } attempt: 1
I20260619 02:42:09.718914 19544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:09.718764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.124:35771 (local address 127.0.0.1:42662)
0619 02:42:09.718837 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:09.718873 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.124:35771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
Jun 19 02:42:09 dist-test-slave-zpbg krb5kdc[19386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836929, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for kudu/127.5.241.125@KRBTEST.COM
I20260619 02:42:09.719843 19544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:09.719186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.124:35771 (local address 127.0.0.1:42678)
0619 02:42:09.719775 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:09.719815 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.124:35771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":572}
W20260619 02:42:09.719944 19542 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.241.124:35771: Network error: Client connection negotiation failed: client connection to 127.5.241.124:35771: connect: Connection refused (error 111)
I20260619 02:42:09.722922 19546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:09.715395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36968 (local address 127.5.241.125:43445)
0619 02:42:09.715451 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:09.715454 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:09.715472 (+ 18us) server_negotiation.cc:407] Connection header received
0619 02:42:09.715504 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:09.715505 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:09.715516 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:09.715598 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:42:09.715962 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:09.716756 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:09.717500 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:09.717821 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:09.717894 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:09.720126 (+ 2232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:42:09.720127 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:42:09.720128 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:42:09.720135 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:42:09.721736 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:09.722099 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:09.722100 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:09.722101 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:09.722162 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:09.722367 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:09.722368 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:09.722368 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:09.722498 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:42:09.722575 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:09.722734 (+ 159us) server_negotiation.cc:299] Negotiation successful
0619 02:42:09.722838 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 02:42:09.722927 19469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:09.715227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.125:43445 (local address 127.0.0.1:36968)
0619 02:42:09.715323 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:09.715336 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:09.715415 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:09.715636 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:09.715638 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:09.715795 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:42:09.715860 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:09.715862 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:09.716857 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:09.716859 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:09.717411 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:09.717413 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:09.717922 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:09.717923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:09.717993 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:09.718539 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:42:09.718544 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:42:09.720034 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:42:09.721840 (+ 1806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:09.721841 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:09.721841 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:09.722012 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:09.722228 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:09.722229 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:09.722230 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:09.722294 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:09.722568 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:42:09.722569 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:42:09.722618 (+ 49us) client_negotiation.cc:769] Sending connection context
0619 02:42:09.722708 (+ 90us) client_negotiation.cc:240] Negotiation successful
0619 02:42:09.722828 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260619 02:42:09.723426 19467 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.241.124" port: 35771 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:42:09.723824 19469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:09.723712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.124:35771 (local address 127.0.0.1:42690)
0619 02:42:09.723765 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:09.723798 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.124:35771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260619 02:42:09.724149 19469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:09.724044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.124:35771 (local address 127.0.0.1:42692)
0619 02:42:09.724107 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:09.724132 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.124:35771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260619 02:42:09.724232 19467 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.241.124:35771: Network error: Client connection negotiation failed: client connection to 127.5.241.124:35771: connect: Connection refused (error 111)
I20260619 02:42:09.755383 19467 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.241.124" port: 35771 } attempt: 1
I20260619 02:42:09.755932 19469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:09.755781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.124:35771 (local address 127.0.0.1:42696)
0619 02:42:09.755854 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:09.755899 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.124:35771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260619 02:42:09.756340 19469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:09.756230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.124:35771 (local address 127.0.0.1:42700)
0619 02:42:09.756284 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:09.756314 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.124:35771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260619 02:42:09.756449 19467 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.241.124:35771: Network error: Client connection negotiation failed: client connection to 127.5.241.124:35771: connect: Connection refused (error 111)
W20260619 02:42:09.760564 19549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:42:09.760761 19549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:42:09.760783 19549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:42:09.761862 19549 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:42:09.761896 19549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:42:09.761910 19549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:42:09.761924 19549 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:42:09.761935 19549 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:42:09.763484 19549 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:38957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:33863,127.5.241.125:43445,127.5.241.124:35771
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.124.keytab
--principal=kudu/127.5.241.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.124:35771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.241.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19549
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:42:09.763667 19549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:42:09.763844 19549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:42:09.768586 19554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:42:09.768814 19549 server_base.cc:1034] running on GCE node
W20260619 02:42:09.768586 19557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:09.768569 19555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:42:09.769577 19549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:42:09.769905 19549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:42:09.771087 19549 hybrid_clock.cc:648] HybridClock initialized: now 1781836929771062 us; error 35 us; skew 500 ppm
Jun 19 02:42:09 dist-test-slave-zpbg krb5kdc[19386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836929, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:42:09.773777 19549 init.cc:377] Logged in from keytab as kudu/127.5.241.124@KRBTEST.COM (short username kudu)
I20260619 02:42:09.774152 19549 webserver.cc:460] Webserver started at http://127.5.241.124:37315/ using document root <none> and password file <none>
I20260619 02:42:09.774397 19549 fs_manager.cc:359] Metadata directory not provided
I20260619 02:42:09.774442 19549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:42:09.774577 19549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:42:09.775699 19549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/minicluster-data/master-2/data/instance:
uuid: "4a91feb0b6984d39abb13bddde68a10b"
format_stamp: "Formatted at 2026-06-19 02:42:09 on dist-test-slave-zpbg"
server_key: "3487e5ff2320ed8dc0bf46d886b1a3b8"
server_key_iv: "cd22ca85785cfab8ba232fb057bf5712"
server_key_version: "encryptionkey@0"
I20260619 02:42:09.776067 19549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/minicluster-data/master-2/wal/instance:
uuid: "4a91feb0b6984d39abb13bddde68a10b"
format_stamp: "Formatted at 2026-06-19 02:42:09 on dist-test-slave-zpbg"
server_key: "3487e5ff2320ed8dc0bf46d886b1a3b8"
server_key_iv: "cd22ca85785cfab8ba232fb057bf5712"
server_key_version: "encryptionkey@0"
I20260619 02:42:09.777359 19549 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:42:09.778067 19564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:09.778285 19549 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:42:09.778357 19549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/minicluster-data/master-2/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/minicluster-data/master-2/wal
uuid: "4a91feb0b6984d39abb13bddde68a10b"
format_stamp: "Formatted at 2026-06-19 02:42:09 on dist-test-slave-zpbg"
server_key: "3487e5ff2320ed8dc0bf46d886b1a3b8"
server_key_iv: "cd22ca85785cfab8ba232fb057bf5712"
server_key_version: "encryptionkey@0"
I20260619 02:42:09.778435 19549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:42:09.788110 19542 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.241.124" port: 35771 } attempt: 2
I20260619 02:42:09.788542 19544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:09.788416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.124:35771 (local address 127.0.0.1:42716)
0619 02:42:09.788473 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:09.788512 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.124:35771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260619 02:42:09.788877 19544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:09.788803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.124:35771 (local address 127.0.0.1:42726)
0619 02:42:09.788840 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:09.788861 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.124:35771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260619 02:42:09.788992 19542 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.241.124:35771: Network error: Client connection negotiation failed: client connection to 127.5.241.124:35771: connect: Connection refused (error 111)
I20260619 02:42:09.804152 19549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:42:09.806257 19549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:42:09.806416 19549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:42:09.810206 19549 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.124:35771
I20260619 02:42:09.810251 19616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.124:35771 every 8 connection(s)
I20260619 02:42:09.810623 19549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/minicluster-data/master-2/data/info.pb
I20260619 02:42:09.811520 19617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:42:09.812776 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 19549
I20260619 02:42:09.812917 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836772852275-6085-0/minicluster-data/master-2/wal/instance
I20260619 02:42:09.813113 6085 external_mini_cluster.cc:1428] Setting key 1eadcfd5090ac7a7ea956cf2ac9b8992
I20260619 02:42:09.813140 19617 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.241.126" port: 33863 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:42:09 dist-test-slave-zpbg krb5kdc[19386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836929, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.124@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
Jun 19 02:42:09 dist-test-slave-zpbg krb5kdc[19386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836929, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.241.124@KRBTEST.COM
I20260619 02:42:09.822865 19471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:09.813956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56778 (local address 127.5.241.126:33863)
0619 02:42:09.814002 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:09.814004 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:09.814129 (+ 125us) server_negotiation.cc:407] Connection header received
0619 02:42:09.814517 (+ 388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:09.814518 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:09.814531 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:09.814585 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:42:09.815228 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:09.815854 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:09.816732 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:09.816934 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:09.817022 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:09.819296 (+ 2274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:42:09.819298 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:42:09.819298 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:42:09.819306 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:42:09.821186 (+ 1880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:09.821818 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:09.821819 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:09.821819 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:09.821870 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:09.822157 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:09.822158 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:09.822158 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:09.822314 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:42:09.822385 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:09.822638 (+ 253us) server_negotiation.cc:299] Negotiation successful
0619 02:42:09.822736 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 02:42:09.823100 19621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:09.813857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:33863 (local address 127.0.0.1:56778)
0619 02:42:09.814001 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:09.814050 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:09.814418 (+ 368us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:09.814604 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:09.814607 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:09.814964 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:42:09.815122 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:09.815127 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:09.815933 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:09.815934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:09.816620 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:09.816622 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:09.817031 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:09.817032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:09.817160 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:09.817623 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:42:09.817630 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:42:09.819203 (+ 1573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:42:09.821294 (+ 2091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:09.821301 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:09.821303 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:09.821709 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:09.821934 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:09.821935 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:09.821935 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:09.822071 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:09.822387 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:42:09.822402 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:42:09.822493 (+ 91us) client_negotiation.cc:769] Sending connection context
0619 02:42:09.822603 (+ 110us) client_negotiation.cc:240] Negotiation successful
0619 02:42:09.822813 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":7,"thread_start_us":85,"threads_started":1}
I20260619 02:42:09.823103 19620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:09.813704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42740 (local address 127.5.241.124:35771)
0619 02:42:09.813945 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:09.813954 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:09.813984 (+ 30us) server_negotiation.cc:407] Connection header received
0619 02:42:09.814400 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:09.814407 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:09.814426 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:09.814538 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:42:09.815189 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:09.816038 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:09.816783 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:09.817133 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:09.817216 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:09.819837 (+ 2621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:42:09.819845 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:42:09.819845 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:42:09.819853 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:42:09.821615 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:09.822003 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:09.822005 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:09.822006 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:09.822067 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:09.822263 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:09.822264 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:09.822264 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:09.822504 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:42:09.822645 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:09.822727 (+ 82us) server_negotiation.cc:299] Negotiation successful
0619 02:42:09.822861 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":82,"server-negotiator.queue_time_us":194,"thread_start_us":102,"threads_started":1}
I20260619 02:42:09.823675 19617 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.241.125" port: 43445 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:42:09 dist-test-slave-zpbg krb5kdc[19386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836929, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.124@KRBTEST.COM for kudu/127.5.241.125@KRBTEST.COM
Jun 19 02:42:09 dist-test-slave-zpbg krb5kdc[19386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836929, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:42:09.831168 19621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:09.823946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.125:43445 (local address 127.0.0.1:36978)
0619 02:42:09.824038 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:09.824047 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:09.824091 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:09.824227 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:09.824228 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:09.824378 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:42:09.824428 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:09.824430 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:09.825264 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:09.825264 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:09.826015 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:09.826017 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:09.826471 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:09.826472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:09.826528 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:09.827005 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:42:09.827009 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:42:09.828346 (+ 1337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:42:09.830161 (+ 1815us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:09.830162 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:09.830162 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:09.830326 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:09.830533 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:09.830533 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:09.830534 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:09.830581 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:09.830871 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:42:09.830872 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:42:09.830914 (+ 42us) client_negotiation.cc:769] Sending connection context
0619 02:42:09.830989 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 02:42:09.831085 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260619 02:42:09.831220 19546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:09.823998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36978 (local address 127.5.241.125:43445)
0619 02:42:09.824049 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:09.824051 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:09.824079 (+ 28us) server_negotiation.cc:407] Connection header received
0619 02:42:09.824144 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:09.824145 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:09.824164 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:09.824222 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:42:09.824507 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:09.825187 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:09.826111 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:09.826378 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:09.826455 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:09.828449 (+ 1994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:42:09.828450 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:42:09.828450 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:42:09.828457 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:42:09.830078 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:09.830400 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:09.830401 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:09.830402 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:09.830464 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:09.830642 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:09.830643 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:09.830643 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:09.830806 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:42:09.830866 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:09.831016 (+ 150us) server_negotiation.cc:299] Negotiation successful
0619 02:42:09.831103 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 02:42:09.831573 19467 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.241.124" port: 35771 } attempt: 2
I20260619 02:42:09.831723 19617 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.241.124" port: 35771 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:42:09.832408 19471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:09.823954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56782 (local address 127.5.241.126:33863)
0619 02:42:09.823988 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:09.823990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:09.824041 (+ 51us) server_negotiation.cc:407] Connection header received
0619 02:42:09.824116 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:09.824117 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:09.824128 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:09.824184 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:42:09.824714 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:09.825355 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:09.826251 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:09.826536 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:09.826594 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:09.829095 (+ 2501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:42:09.829096 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:42:09.829096 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:42:09.829103 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:42:09.830679 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:09.831096 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:09.831097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:09.831097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:09.831151 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:09.831435 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:09.831436 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:09.831436 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:09.831598 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:42:09.831786 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:09.832214 (+ 428us) server_negotiation.cc:299] Negotiation successful
0619 02:42:09.832313 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
Jun 19 02:42:09 dist-test-slave-zpbg krb5kdc[19386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836929, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.124@KRBTEST.COM for kudu/127.5.241.124@KRBTEST.COM
Jun 19 02:42:09 dist-test-slave-zpbg krb5kdc[19386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836929, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for kudu/127.5.241.124@KRBTEST.COM
Jun 19 02:42:09 dist-test-slave-zpbg krb5kdc[19386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836929, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.241.125@KRBTEST.COM
I20260619 02:42:09.840622 19469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:09.831810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.124:35771 (local address 127.0.0.1:42746)
0619 02:42:09.831856 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:09.831865 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:09.831901 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:09.832050 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:09.832052 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:09.832198 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:42:09.832232 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:09.832233 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:09.833181 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:09.833182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:09.833960 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:09.833962 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:09.834987 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:09.834988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:09.835048 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:09.835426 (+ 378us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:42:09.835431 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:42:09.837345 (+ 1914us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:42:09.839560 (+ 2215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:09.839561 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:09.839562 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:09.839737 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:09.839958 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:09.839959 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:09.839960 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:09.840024 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:09.840268 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:42:09.840269 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:42:09.840327 (+ 58us) client_negotiation.cc:769] Sending connection context
0619 02:42:09.840399 (+ 72us) client_negotiation.cc:240] Negotiation successful
0619 02:42:09.840499 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 02:42:09.840655 19620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:09.831882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42746 (local address 127.5.241.124:35771)
0619 02:42:09.831917 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:09.831919 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:09.831926 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:42:09.831947 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:09.831947 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:09.831957 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:09.832314 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:42:09.832342 (+ 28us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:09.833093 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:09.834046 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:09.834891 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:09.835200 (+ 309us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:09.837433 (+ 2233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:42:09.837434 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:42:09.837434 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:42:09.837440 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:42:09.839454 (+ 2014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:09.839832 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:09.839833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:09.839833 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:09.839889 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:09.840089 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:09.840090 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:09.840091 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:09.840199 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:42:09.840274 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:09.840429 (+ 155us) server_negotiation.cc:299] Negotiation successful
0619 02:42:09.840547 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 02:42:09.841562 19546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:09.833175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36994 (local address 127.5.241.125:43445)
0619 02:42:09.833213 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:09.833215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:09.833223 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:42:09.833272 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:09.833273 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:09.833282 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:09.833340 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:42:09.833771 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:09.834204 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:09.835034 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:09.835233 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:09.835315 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:09.838031 (+ 2716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:42:09.838032 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:42:09.838032 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:42:09.838039 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:42:09.839932 (+ 1893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:09.840341 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:09.840343 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:09.840343 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:09.840400 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:09.840611 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:09.840612 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:09.840612 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:09.840802 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:42:09.840994 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:09.841197 (+ 203us) server_negotiation.cc:299] Negotiation successful
0619 02:42:09.841463 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:42:09.841668 19621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:09.831956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.124:35771 (local address 127.0.0.1:42760)
0619 02:42:09.832018 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:09.832027 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:09.832054 (+ 27us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:09.832335 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:09.832336 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:09.832465 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:42:09.832486 (+ 21us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:09.832488 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:09.833385 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:09.833386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:09.834098 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:09.834099 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:09.834579 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:09.834581 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:09.834639 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:09.835375 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:42:09.835380 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:42:09.837890 (+ 2510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:42:09.840694 (+ 2804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:09.840694 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:09.840695 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:09.840866 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:09.841084 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:09.841085 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:09.841085 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:09.841136 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:09.841406 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:42:09.841407 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:42:09.841443 (+ 36us) client_negotiation.cc:769] Sending connection context
0619 02:42:09.841505 (+ 62us) client_negotiation.cc:240] Negotiation successful
0619 02:42:09.841587 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260619 02:42:09.842015 19622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:09.832027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42760 (local address 127.5.241.124:35771)
0619 02:42:09.832195 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:09.832196 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:09.832203 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:42:09.832222 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:09.832222 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:09.832231 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:09.832305 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:42:09.832562 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:09.833300 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:09.834163 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:09.834501 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:09.834571 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:09.837965 (+ 3394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:42:09.837967 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:42:09.837967 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:42:09.837974 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:42:09.840617 (+ 2643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:09.840951 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:09.840952 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:09.840953 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:09.841009 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:09.841204 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:09.841205 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:09.841205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:09.841336 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:42:09.841414 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:09.841820 (+ 406us) server_negotiation.cc:299] Negotiation successful
0619 02:42:09.841891 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":120,"threads_started":1}
I20260619 02:42:09.843118 6085 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:42:09.843215 19467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd6fe680bd2441dfb41352c041263edc: Bootstrap starting.
I20260619 02:42:09.843420 19617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a91feb0b6984d39abb13bddde68a10b: Bootstrap starting.
I20260619 02:42:09.843981 19467 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bd6fe680bd2441dfb41352c041263edc: Neither blocks nor log segments found. Creating new log.
I20260619 02:42:09.843998 19617 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a91feb0b6984d39abb13bddde68a10b: Neither blocks nor log segments found. Creating new log.
I20260619 02:42:09.844306 19617 log.cc:824] T 00000000000000000000000000000000 P 4a91feb0b6984d39abb13bddde68a10b: Log is configured to *not* fsync() on all Append() calls
I20260619 02:42:09.844306 19467 log.cc:824] T 00000000000000000000000000000000 P bd6fe680bd2441dfb41352c041263edc: Log is configured to *not* fsync() on all Append() calls
I20260619 02:42:09.845619 19467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd6fe680bd2441dfb41352c041263edc: No bootstrap required, opened a new log
I20260619 02:42:09.845750 19617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a91feb0b6984d39abb13bddde68a10b: No bootstrap required, opened a new log
I20260619 02:42:09.847671 19467 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd6fe680bd2441dfb41352c041263edc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd6fe680bd2441dfb41352c041263edc" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 33863 } } peers { permanent_uuid: "15f29de8c8094db1a3745c20e37ef503" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 43445 } } peers { permanent_uuid: "4a91feb0b6984d39abb13bddde68a10b" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 35771 } }
I20260619 02:42:09.847673 19617 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a91feb0b6984d39abb13bddde68a10b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd6fe680bd2441dfb41352c041263edc" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 33863 } } peers { permanent_uuid: "15f29de8c8094db1a3745c20e37ef503" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 43445 } } peers { permanent_uuid: "4a91feb0b6984d39abb13bddde68a10b" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 35771 } }
I20260619 02:42:09.847827 19617 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4a91feb0b6984d39abb13bddde68a10b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:42:09.847827 19467 raft_consensus.cc:374] T 00000000000000000000000000000000 P bd6fe680bd2441dfb41352c041263edc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:42:09.847865 19467 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd6fe680bd2441dfb41352c041263edc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd6fe680bd2441dfb41352c041263edc, State: Initialized, Role: FOLLOWER
I20260619 02:42:09.847865 19617 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a91feb0b6984d39abb13bddde68a10b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a91feb0b6984d39abb13bddde68a10b, State: Initialized, Role: FOLLOWER
I20260619 02:42:09.848006 19467 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd6fe680bd2441dfb41352c041263edc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd6fe680bd2441dfb41352c041263edc" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 33863 } } peers { permanent_uuid: "15f29de8c8094db1a3745c20e37ef503" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 43445 } } peers { permanent_uuid: "4a91feb0b6984d39abb13bddde68a10b" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 35771 } }
I20260619 02:42:09.848011 19617 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a91feb0b6984d39abb13bddde68a10b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd6fe680bd2441dfb41352c041263edc" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 33863 } } peers { permanent_uuid: "15f29de8c8094db1a3745c20e37ef503" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 43445 } } peers { permanent_uuid: "4a91feb0b6984d39abb13bddde68a10b" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 35771 } }
I20260619 02:42:09.848276 19632 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd6fe680bd2441dfb41352c041263edc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd6fe680bd2441dfb41352c041263edc" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 33863 } } peers { permanent_uuid: "15f29de8c8094db1a3745c20e37ef503" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 43445 } } peers { permanent_uuid: "4a91feb0b6984d39abb13bddde68a10b" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 35771 } } }
I20260619 02:42:09.848321 19617 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a91feb0b6984d39abb13bddde68a10b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:42:09.848322 19467 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd6fe680bd2441dfb41352c041263edc [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:42:09.849725 19633 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a91feb0b6984d39abb13bddde68a10b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd6fe680bd2441dfb41352c041263edc" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 33863 } } peers { permanent_uuid: "15f29de8c8094db1a3745c20e37ef503" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 43445 } } peers { permanent_uuid: "4a91feb0b6984d39abb13bddde68a10b" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 35771 } } }
W20260619 02:42:09.850639 19655 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4a91feb0b6984d39abb13bddde68a10b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260619 02:42:09.851253 19633 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a91feb0b6984d39abb13bddde68a10b [sys.catalog]: This master's current role is: FOLLOWER
W20260619 02:42:09.851262 19655 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:42:09.849807 19632 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd6fe680bd2441dfb41352c041263edc [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:42:09.850939 19399 master_runner.cc:186] Error getting master registration for 127.5.241.125:43445: OK, instance_id { permanent_uuid: "15f29de8c8094db1a3745c20e37ef503" instance_seqno: 1781836929644991 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 02:42:09.853668 19399 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260619 02:42:09.850144 19654 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bd6fe680bd2441dfb41352c041263edc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:42:09.855086 19654 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:42:09.855588 19549 master_runner.cc:186] Error getting master registration for 127.5.241.125:43445: OK, instance_id { permanent_uuid: "15f29de8c8094db1a3745c20e37ef503" instance_seqno: 1781836929644991 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 02:42:09.857841 19549 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 02:42:09.859594 19471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:09.844758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56794 (local address 127.5.241.126:33863)
0619 02:42:09.844794 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:09.844796 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:09.844911 (+ 115us) server_negotiation.cc:407] Connection header received
0619 02:42:09.844978 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:09.844979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:09.844989 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:09.845037 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:42:09.850363 (+ 5326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:09.852200 (+ 1837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:09.853148 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:09.853432 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:09.853475 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:09.855504 (+ 2029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:42:09.855505 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:42:09.855506 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:42:09.855512 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:42:09.857160 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:09.858433 (+ 1273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:09.858435 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:09.858435 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:09.858496 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:09.858705 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:09.858706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:09.858706 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:09.858862 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:42:09.858955 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:09.859381 (+ 426us) server_negotiation.cc:299] Negotiation successful
0619 02:42:09.859471 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:42:09.860237 19546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:09.844524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37000 (local address 127.5.241.125:43445)
0619 02:42:09.844572 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:09.844574 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:09.844919 (+ 345us) server_negotiation.cc:407] Connection header received
0619 02:42:09.845017 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:09.845018 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:09.845030 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:09.845058 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:42:09.850219 (+ 5161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:09.851150 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:09.852155 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:09.852459 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:09.852528 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:09.854126 (+ 1598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:42:09.854127 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:42:09.854127 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:42:09.854135 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:42:09.858626 (+ 4491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:09.859039 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:09.859040 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:09.859041 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:09.859097 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:09.859286 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:09.859286 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:09.859287 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:09.859423 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:42:09.859519 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:09.859930 (+ 411us) server_negotiation.cc:299] Negotiation successful
0619 02:42:09.860142 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 02:42:09.866554 19622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:09.844668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42776 (local address 127.5.241.124:35771)
0619 02:42:09.844704 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:09.844706 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:09.844775 (+ 69us) server_negotiation.cc:407] Connection header received
0619 02:42:09.844841 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:09.844842 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:09.844853 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:09.844908 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:42:09.857565 (+ 12657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:09.858327 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:09.861422 (+ 3095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:09.861737 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:09.862865 (+ 1128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:09.863301 (+ 436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:42:09.863313 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:42:09.863313 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:42:09.863321 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:42:09.865033 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:09.865447 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:09.865448 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:09.865448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:09.865506 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:09.865699 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:09.865700 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:09.865700 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:09.865823 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:42:09.866336 (+ 513us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:09.866377 (+ 41us) server_negotiation.cc:299] Negotiation successful
0619 02:42:09.866459 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 02:42:09.880162 19542 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.241.124" port: 35771 } attempt: 3
Jun 19 02:42:09 dist-test-slave-zpbg krb5kdc[19386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836929, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.125@KRBTEST.COM for kudu/127.5.241.124@KRBTEST.COM
I20260619 02:42:09.888795 19544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:09.880463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.124:35771 (local address 127.0.0.1:42778)
0619 02:42:09.880511 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:09.880522 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:09.880566 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:09.880738 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:09.880739 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:09.880923 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:42:09.880975 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:09.880977 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:09.882435 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:09.882436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:09.883276 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:09.883279 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:09.883765 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:09.883766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:09.883827 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:09.884324 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:42:09.884329 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:42:09.885841 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:42:09.887729 (+ 1888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:09.887730 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:09.887730 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:09.887939 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:09.888141 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:09.888142 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:09.888142 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:09.888194 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:09.888467 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:42:09.888468 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:42:09.888519 (+ 51us) client_negotiation.cc:769] Sending connection context
0619 02:42:09.888597 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 02:42:09.888690 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260619 02:42:09.888806 19622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:09.880525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42778 (local address 127.5.241.124:35771)
0619 02:42:09.880562 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:09.880564 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:09.880572 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:42:09.880620 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:09.880621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:09.880658 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:09.880722 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:42:09.881053 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:09.882347 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:09.883387 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:09.883685 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:09.883760 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:09.885930 (+ 2170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:42:09.885931 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:42:09.885932 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:42:09.885938 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:42:09.887626 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:09.888020 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:09.888021 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:09.888021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:09.888072 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:09.888275 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:09.888276 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:09.888276 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:09.888393 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:42:09.888457 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:09.888636 (+ 179us) server_negotiation.cc:299] Negotiation successful
0619 02:42:09.888712 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 02:42:09.890551 19542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15f29de8c8094db1a3745c20e37ef503: Bootstrap starting.
I20260619 02:42:09.891363 19542 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 15f29de8c8094db1a3745c20e37ef503: Neither blocks nor log segments found. Creating new log.
I20260619 02:42:09.891670 19542 log.cc:824] T 00000000000000000000000000000000 P 15f29de8c8094db1a3745c20e37ef503: Log is configured to *not* fsync() on all Append() calls
I20260619 02:42:09.892539 19542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15f29de8c8094db1a3745c20e37ef503: No bootstrap required, opened a new log
I20260619 02:42:09.894493 19542 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15f29de8c8094db1a3745c20e37ef503 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd6fe680bd2441dfb41352c041263edc" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 33863 } } peers { permanent_uuid: "15f29de8c8094db1a3745c20e37ef503" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 43445 } } peers { permanent_uuid: "4a91feb0b6984d39abb13bddde68a10b" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 35771 } }
I20260619 02:42:09.894631 19542 raft_consensus.cc:374] T 00000000000000000000000000000000 P 15f29de8c8094db1a3745c20e37ef503 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:42:09.894665 19542 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15f29de8c8094db1a3745c20e37ef503 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15f29de8c8094db1a3745c20e37ef503, State: Initialized, Role: FOLLOWER
I20260619 02:42:09.894867 19542 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15f29de8c8094db1a3745c20e37ef503 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd6fe680bd2441dfb41352c041263edc" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 33863 } } peers { permanent_uuid: "15f29de8c8094db1a3745c20e37ef503" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 43445 } } peers { permanent_uuid: "4a91feb0b6984d39abb13bddde68a10b" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 35771 } }
I20260619 02:42:09.895180 19542 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15f29de8c8094db1a3745c20e37ef503 [sys.catalog]: configured and running, proceeding with master startup.
W20260619 02:42:09.896936 19670 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 15f29de8c8094db1a3745c20e37ef503: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:42:09.897032 19670 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:42:09.895139 19659 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15f29de8c8094db1a3745c20e37ef503 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd6fe680bd2441dfb41352c041263edc" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 33863 } } peers { permanent_uuid: "15f29de8c8094db1a3745c20e37ef503" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 43445 } } peers { permanent_uuid: "4a91feb0b6984d39abb13bddde68a10b" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 35771 } } }
I20260619 02:42:09.897260 19659 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15f29de8c8094db1a3745c20e37ef503 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:42:09.899412 19474 master_runner.cc:263] No leader master found from master 15f29de8c8094db1a3745c20e37ef503
I20260619 02:42:09.899458 19474 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260619 02:42:09.985028 19420 tablet.cc:2369] T 00000000000000000000000000000000 P bd6fe680bd2441dfb41352c041263edc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:42:09.994318 19633 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a91feb0b6984d39abb13bddde68a10b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:42:09.994467 19633 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a91feb0b6984d39abb13bddde68a10b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd6fe680bd2441dfb41352c041263edc" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 33863 } } peers { permanent_uuid: "15f29de8c8094db1a3745c20e37ef503" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 43445 } } peers { permanent_uuid: "4a91feb0b6984d39abb13bddde68a10b" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 35771 } }
I20260619 02:42:09.994735 19633 leader_election.cc:288] T 00000000000000000000000000000000 P 4a91feb0b6984d39abb13bddde68a10b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd6fe680bd2441dfb41352c041263edc (127.5.241.126:33863), 15f29de8c8094db1a3745c20e37ef503 (127.5.241.125:43445)
I20260619 02:42:09.994985 19441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4a91feb0b6984d39abb13bddde68a10b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd6fe680bd2441dfb41352c041263edc" is_pre_election: true
I20260619 02:42:09.995138 19441 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bd6fe680bd2441dfb41352c041263edc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a91feb0b6984d39abb13bddde68a10b in term 0.
I20260619 02:42:09.995128 19516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4a91feb0b6984d39abb13bddde68a10b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15f29de8c8094db1a3745c20e37ef503" is_pre_election: true
I20260619 02:42:09.995250 19516 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 15f29de8c8094db1a3745c20e37ef503 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a91feb0b6984d39abb13bddde68a10b in term 0.
I20260619 02:42:09.995447 19568 leader_election.cc:302] T 00000000000000000000000000000000 P 4a91feb0b6984d39abb13bddde68a10b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15f29de8c8094db1a3745c20e37ef503, 4a91feb0b6984d39abb13bddde68a10b; no voters:
I20260619 02:42:09.995623 19633 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a91feb0b6984d39abb13bddde68a10b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:42:09.995677 19633 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a91feb0b6984d39abb13bddde68a10b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:42:09.995708 19633 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a91feb0b6984d39abb13bddde68a10b [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:42:09.996592 19633 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a91feb0b6984d39abb13bddde68a10b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd6fe680bd2441dfb41352c041263edc" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 33863 } } peers { permanent_uuid: "15f29de8c8094db1a3745c20e37ef503" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 43445 } } peers { permanent_uuid: "4a91feb0b6984d39abb13bddde68a10b" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 35771 } }
I20260619 02:42:09.996734 19633 leader_election.cc:288] T 00000000000000000000000000000000 P 4a91feb0b6984d39abb13bddde68a10b [CANDIDATE]: Term 1 election: Requested vote from peers bd6fe680bd2441dfb41352c041263edc (127.5.241.126:33863), 15f29de8c8094db1a3745c20e37ef503 (127.5.241.125:43445)
I20260619 02:42:09.996874 19441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4a91feb0b6984d39abb13bddde68a10b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd6fe680bd2441dfb41352c041263edc"
I20260619 02:42:09.996894 19516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4a91feb0b6984d39abb13bddde68a10b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15f29de8c8094db1a3745c20e37ef503"
I20260619 02:42:09.996941 19441 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd6fe680bd2441dfb41352c041263edc [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:42:09.996956 19516 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15f29de8c8094db1a3745c20e37ef503 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:42:09.997735 19441 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bd6fe680bd2441dfb41352c041263edc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a91feb0b6984d39abb13bddde68a10b in term 1.
I20260619 02:42:09.997735 19516 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 15f29de8c8094db1a3745c20e37ef503 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a91feb0b6984d39abb13bddde68a10b in term 1.
I20260619 02:42:09.997892 19568 leader_election.cc:302] T 00000000000000000000000000000000 P 4a91feb0b6984d39abb13bddde68a10b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15f29de8c8094db1a3745c20e37ef503, 4a91feb0b6984d39abb13bddde68a10b; no voters:
I20260619 02:42:09.997985 19633 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a91feb0b6984d39abb13bddde68a10b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:42:09.998169 19633 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a91feb0b6984d39abb13bddde68a10b [term 1 LEADER]: Becoming Leader. State: Replica: 4a91feb0b6984d39abb13bddde68a10b, State: Running, Role: LEADER
I20260619 02:42:09.998252 19633 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a91feb0b6984d39abb13bddde68a10b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd6fe680bd2441dfb41352c041263edc" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 33863 } } peers { permanent_uuid: "15f29de8c8094db1a3745c20e37ef503" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 43445 } } peers { permanent_uuid: "4a91feb0b6984d39abb13bddde68a10b" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 35771 } }
I20260619 02:42:09.998610 19672 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a91feb0b6984d39abb13bddde68a10b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a91feb0b6984d39abb13bddde68a10b. Latest consensus state: current_term: 1 leader_uuid: "4a91feb0b6984d39abb13bddde68a10b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd6fe680bd2441dfb41352c041263edc" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 33863 } } peers { permanent_uuid: "15f29de8c8094db1a3745c20e37ef503" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 43445 } } peers { permanent_uuid: "4a91feb0b6984d39abb13bddde68a10b" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 35771 } } }
I20260619 02:42:09.998745 19672 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a91feb0b6984d39abb13bddde68a10b [sys.catalog]: This master's current role is: LEADER
I20260619 02:42:09.999176 19674 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:42:09.999358 19674 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:42:10.000571 19516 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 15f29de8c8094db1a3745c20e37ef503 [term 1 FOLLOWER]: Refusing update from remote peer 4a91feb0b6984d39abb13bddde68a10b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:42:10.000591 19441 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bd6fe680bd2441dfb41352c041263edc [term 1 FOLLOWER]: Refusing update from remote peer 4a91feb0b6984d39abb13bddde68a10b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:42:10.000874 19672 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4a91feb0b6984d39abb13bddde68a10b [LEADER]: Connected to new peer: Peer: permanent_uuid: "15f29de8c8094db1a3745c20e37ef503" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 43445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:42:10.001073 19672 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4a91feb0b6984d39abb13bddde68a10b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd6fe680bd2441dfb41352c041263edc" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 33863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:42:10.002163 19659 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15f29de8c8094db1a3745c20e37ef503 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a91feb0b6984d39abb13bddde68a10b. Latest consensus state: current_term: 1 leader_uuid: "4a91feb0b6984d39abb13bddde68a10b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd6fe680bd2441dfb41352c041263edc" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 33863 } } peers { permanent_uuid: "15f29de8c8094db1a3745c20e37ef503" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 43445 } } peers { permanent_uuid: "4a91feb0b6984d39abb13bddde68a10b" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 35771 } } }
I20260619 02:42:10.002271 19659 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15f29de8c8094db1a3745c20e37ef503 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:42:10.003477 19632 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd6fe680bd2441dfb41352c041263edc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a91feb0b6984d39abb13bddde68a10b. Latest consensus state: current_term: 1 leader_uuid: "4a91feb0b6984d39abb13bddde68a10b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd6fe680bd2441dfb41352c041263edc" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 33863 } } peers { permanent_uuid: "15f29de8c8094db1a3745c20e37ef503" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 43445 } } peers { permanent_uuid: "4a91feb0b6984d39abb13bddde68a10b" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 35771 } } }
I20260619 02:42:10.003587 19632 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd6fe680bd2441dfb41352c041263edc [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:42:10.007887 19675 mvcc.cc:205] Tried to move back new op lower bound from 7298404065280503808 to 7298404065273352192. Current Snapshot: MvccSnapshot[applied={T|T < 7298404065280503808}]
I20260619 02:42:10.007848 19672 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a91feb0b6984d39abb13bddde68a10b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4a91feb0b6984d39abb13bddde68a10b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd6fe680bd2441dfb41352c041263edc" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 33863 } } peers { permanent_uuid: "15f29de8c8094db1a3745c20e37ef503" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 43445 } } peers { permanent_uuid: "4a91feb0b6984d39abb13bddde68a10b" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 35771 } } }
I20260619 02:42:10.008004 19672 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a91feb0b6984d39abb13bddde68a10b [sys.catalog]: This master's current role is: LEADER
I20260619 02:42:10.008543 19659 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15f29de8c8094db1a3745c20e37ef503 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4a91feb0b6984d39abb13bddde68a10b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd6fe680bd2441dfb41352c041263edc" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 33863 } } peers { permanent_uuid: "15f29de8c8094db1a3745c20e37ef503" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 43445 } } peers { permanent_uuid: "4a91feb0b6984d39abb13bddde68a10b" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 35771 } } }
I20260619 02:42:10.008623 19659 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15f29de8c8094db1a3745c20e37ef503 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:42:10.009023 19672 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a91feb0b6984d39abb13bddde68a10b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4a91feb0b6984d39abb13bddde68a10b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd6fe680bd2441dfb41352c041263edc" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 33863 } } peers { permanent_uuid: "15f29de8c8094db1a3745c20e37ef503" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 43445 } } peers { permanent_uuid: "4a91feb0b6984d39abb13bddde68a10b" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 35771 } } }
I20260619 02:42:10.009114 19672 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a91feb0b6984d39abb13bddde68a10b [sys.catalog]: This master's current role is: LEADER
I20260619 02:42:10.009294 19674 catalog_manager.cc:1347] Generated new cluster ID: 31087c8c73cb4ca398840675ad72d387
I20260619 02:42:10.009351 19674 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:42:10.009639 19678 mvcc.cc:205] Tried to move back new op lower bound from 7298404065280503808 to 7298404065273352192. Current Snapshot: MvccSnapshot[applied={T|T < 7298404065280503808}]
I20260619 02:42:10.010851 19632 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd6fe680bd2441dfb41352c041263edc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4a91feb0b6984d39abb13bddde68a10b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd6fe680bd2441dfb41352c041263edc" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 33863 } } peers { permanent_uuid: "15f29de8c8094db1a3745c20e37ef503" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 43445 } } peers { permanent_uuid: "4a91feb0b6984d39abb13bddde68a10b" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 35771 } } }
I20260619 02:42:10.010923 19632 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd6fe680bd2441dfb41352c041263edc [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:42:10.074116 19674 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:42:10.074821 19674 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:42:10.079591 19674 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4a91feb0b6984d39abb13bddde68a10b: Generated new TSK 0
I20260619 02:42:10.079797 19674 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:42:10.855615 19654 catalog_manager.cc:1259] Loaded cluster ID: 31087c8c73cb4ca398840675ad72d387
I20260619 02:42:10.855693 19654 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bd6fe680bd2441dfb41352c041263edc: loading cluster ID for follower catalog manager: success
I20260619 02:42:10.856456 19654 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bd6fe680bd2441dfb41352c041263edc: acquiring CA information for follower catalog manager: success
I20260619 02:42:10.856702 19654 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bd6fe680bd2441dfb41352c041263edc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 02:42:10.897488 19670 catalog_manager.cc:1259] Loaded cluster ID: 31087c8c73cb4ca398840675ad72d387
I20260619 02:42:10.897565 19670 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 15f29de8c8094db1a3745c20e37ef503: loading cluster ID for follower catalog manager: success
I20260619 02:42:10.898370 19670 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 15f29de8c8094db1a3745c20e37ef503: acquiring CA information for follower catalog manager: success
I20260619 02:42:10.898651 19670 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 15f29de8c8094db1a3745c20e37ef503: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 02:42:11.129977 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 19399
I20260619 02:42:11.135196 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 19474
I20260619 02:42:11.140018 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 19549
2026-06-19T02:42:11Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2933 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836772852275-6085-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:42:11 dist-test-slave-zpbg krb5kdc[19689](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:42:11 dist-test-slave-zpbg krb5kdc[19689](info): set up 2 sockets
Jun 19 02:42:11 dist-test-slave-zpbg krb5kdc[19689](info): commencing operation
krb5kdc: starting...
W20260619 02:42:12.178845 6085 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Jun 19 02:42:12 dist-test-slave-zpbg krb5kdc[19689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836932, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:42:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:42:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.241.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
I20260619 02:42:12.301637 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:39633
--webserver_interface=127.5.241.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:35591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:39633
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:42:12.382135 19702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:42:12.382383 19702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:42:12.382441 19702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:42:12.383638 19702 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:42:12.383708 19702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:42:12.383733 19702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:42:12.383764 19702 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:42:12.383791 19702 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:42:12.385247 19702 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:35591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836772852275-6085-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:39633
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:39633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.241.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:42:12.385515 19702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:42:12.385751 19702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:42:12.390939 19710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:12.390964 19708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:12.390939 19707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:42:12.390957 19702 server_base.cc:1034] running on GCE node
I20260619 02:42:12.391449 19702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:42:12.391742 19702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:42:12.392894 19702 hybrid_clock.cc:648] HybridClock initialized: now 1781836932392884 us; error 33 us; skew 500 ppm
Jun 19 02:42:12 dist-test-slave-zpbg krb5kdc[19689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836932, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:42:12.395496 19702 init.cc:377] Logged in from keytab as kudu/127.5.241.126@KRBTEST.COM (short username kudu)
I20260619 02:42:12.395951 19702 webserver.cc:460] Webserver started at http://127.5.241.126:34455/ using document root <none> and password file <none>
I20260619 02:42:12.396186 19702 fs_manager.cc:359] Metadata directory not provided
I20260619 02:42:12.396234 19702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:42:12.396348 19702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:42:12.397419 19702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836772852275-6085-0/minicluster-data/master-0/data/instance:
uuid: "01037a76f07c41cda851f38001167801"
format_stamp: "Formatted at 2026-06-19 02:42:12 on dist-test-slave-zpbg"
server_key: "e088c756c0254f475805d2da4830c062"
server_key_iv: "2961b06d39ed0ea7df3546d8b42a3f96"
server_key_version: "encryptionkey@0"
I20260619 02:42:12.397784 19702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836772852275-6085-0/minicluster-data/master-0/wal/instance:
uuid: "01037a76f07c41cda851f38001167801"
format_stamp: "Formatted at 2026-06-19 02:42:12 on dist-test-slave-zpbg"
server_key: "e088c756c0254f475805d2da4830c062"
server_key_iv: "2961b06d39ed0ea7df3546d8b42a3f96"
server_key_version: "encryptionkey@0"
I20260619 02:42:12.399187 19702 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:42:12.399988 19717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:12.400179 19702 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:42:12.400252 19702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "01037a76f07c41cda851f38001167801"
format_stamp: "Formatted at 2026-06-19 02:42:12 on dist-test-slave-zpbg"
server_key: "e088c756c0254f475805d2da4830c062"
server_key_iv: "2961b06d39ed0ea7df3546d8b42a3f96"
server_key_version: "encryptionkey@0"
I20260619 02:42:12.400362 19702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:42:12.413743 19702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:42:12.416071 19702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:42:12.416253 19702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:42:12.420435 19702 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.126:39633
I20260619 02:42:12.420472 19769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.126:39633 every 8 connection(s)
I20260619 02:42:12.420830 19702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:42:12.421681 19770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:42:12.424119 19770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01037a76f07c41cda851f38001167801: Bootstrap starting.
I20260619 02:42:12.424705 19770 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01037a76f07c41cda851f38001167801: Neither blocks nor log segments found. Creating new log.
I20260619 02:42:12.424994 19770 log.cc:824] T 00000000000000000000000000000000 P 01037a76f07c41cda851f38001167801: Log is configured to *not* fsync() on all Append() calls
I20260619 02:42:12.425716 19770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01037a76f07c41cda851f38001167801: No bootstrap required, opened a new log
I20260619 02:42:12.426965 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 19702
I20260619 02:42:12.427085 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836772852275-6085-0/minicluster-data/master-0/wal/instance
I20260619 02:42:12.427281 6085 external_mini_cluster.cc:1428] Setting key caa2ed7cea0f656d722ff8f0621aea48
I20260619 02:42:12.427706 19770 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01037a76f07c41cda851f38001167801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01037a76f07c41cda851f38001167801" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 39633 } }
I20260619 02:42:12.427911 19770 raft_consensus.cc:374] T 00000000000000000000000000000000 P 01037a76f07c41cda851f38001167801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:42:12.428037 19770 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01037a76f07c41cda851f38001167801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01037a76f07c41cda851f38001167801, State: Initialized, Role: FOLLOWER
I20260619 02:42:12.428211 19770 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01037a76f07c41cda851f38001167801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01037a76f07c41cda851f38001167801" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 39633 } }
I20260619 02:42:12.428318 19770 raft_consensus.cc:388] T 00000000000000000000000000000000 P 01037a76f07c41cda851f38001167801 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:42:12.428362 19770 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01037a76f07c41cda851f38001167801 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:42:12.428414 19770 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01037a76f07c41cda851f38001167801 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:42:12.429328 19770 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01037a76f07c41cda851f38001167801 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01037a76f07c41cda851f38001167801" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 39633 } }
I20260619 02:42:12.429437 19770 leader_election.cc:302] T 00000000000000000000000000000000 P 01037a76f07c41cda851f38001167801 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 01037a76f07c41cda851f38001167801; no voters:
I20260619 02:42:12.429579 19770 leader_election.cc:288] T 00000000000000000000000000000000 P 01037a76f07c41cda851f38001167801 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:42:12.429723 19775 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01037a76f07c41cda851f38001167801 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:42:12.429859 19770 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01037a76f07c41cda851f38001167801 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:42:12.429919 19775 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01037a76f07c41cda851f38001167801 [term 1 LEADER]: Becoming Leader. State: Replica: 01037a76f07c41cda851f38001167801, State: Running, Role: LEADER
I20260619 02:42:12.430066 19775 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01037a76f07c41cda851f38001167801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01037a76f07c41cda851f38001167801" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 39633 } }
I20260619 02:42:12.430500 19776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01037a76f07c41cda851f38001167801 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "01037a76f07c41cda851f38001167801" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01037a76f07c41cda851f38001167801" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 39633 } } }
I20260619 02:42:12.430588 19776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01037a76f07c41cda851f38001167801 [sys.catalog]: This master's current role is: LEADER
I20260619 02:42:12.430748 19777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01037a76f07c41cda851f38001167801 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01037a76f07c41cda851f38001167801. Latest consensus state: current_term: 1 leader_uuid: "01037a76f07c41cda851f38001167801" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01037a76f07c41cda851f38001167801" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 39633 } } }
I20260619 02:42:12.430830 19777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01037a76f07c41cda851f38001167801 [sys.catalog]: This master's current role is: LEADER
I20260619 02:42:12.431135 19784 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:42:12.431675 19784 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:42:12.433445 19784 catalog_manager.cc:1347] Generated new cluster ID: 412afdb1ad4a47c7abe7d09a1192a9ad
I20260619 02:42:12.433503 19784 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 02:42:12 dist-test-slave-zpbg krb5kdc[19689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836932, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:42:12.438642 19774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:12.427845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42130 (local address 127.5.241.126:39633)
0619 02:42:12.428041 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:12.428050 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:12.428085 (+ 35us) server_negotiation.cc:407] Connection header received
0619 02:42:12.428213 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:12.428221 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:12.428275 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:12.428414 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:42:12.429233 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:12.430677 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:12.431809 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:12.432227 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:12.432347 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:12.434953 (+ 2606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:42:12.434956 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:42:12.434957 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:42:12.434966 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:42:12.437075 (+ 2109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:12.437453 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:12.437456 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:12.437456 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:12.437575 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:12.437804 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:12.437804 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:12.437805 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:12.438052 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:42:12.438223 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:12.438323 (+ 100us) server_negotiation.cc:299] Negotiation successful
0619 02:42:12.438466 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":90,"threads_started":1}
I20260619 02:42:12.439579 6085 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:42:12.456418 19784 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:42:12.457023 19784 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:42:12.461068 19784 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 01037a76f07c41cda851f38001167801: Generated new TSK 0
I20260619 02:42:12.461274 19784 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:42:12.493477 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 19702
2026-06-19T02:42:12Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1351 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:42:12 dist-test-slave-zpbg krb5kdc[19797](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:42:12 dist-test-slave-zpbg krb5kdc[19797](info): set up 2 sockets
Jun 19 02:42:12 dist-test-slave-zpbg krb5kdc[19797](info): commencing operation
krb5kdc: starting...
W20260619 02:42:13.530962 6085 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Jun 19 02:42:13 dist-test-slave-zpbg krb5kdc[19797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:42:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:42:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.241.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
I20260619 02:42:13.651799 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:45629
--webserver_interface=127.5.241.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:42191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:45629
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:42:13.731459 19810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:42:13.731659 19810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:42:13.731681 19810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:42:13.732750 19810 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:42:13.732795 19810 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:42:13.732808 19810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:42:13.732821 19810 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:42:13.732833 19810 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:42:13.734217 19810 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:42191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:45629
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:45629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.241.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19810
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:42:13.734404 19810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:42:13.734565 19810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:42:13.739287 19815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:13.739326 19818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:13.739284 19816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:42:13.739547 19810 server_base.cc:1034] running on GCE node
I20260619 02:42:13.739886 19810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:42:13.740170 19810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:42:13.741313 19810 hybrid_clock.cc:648] HybridClock initialized: now 1781836933741291 us; error 36 us; skew 500 ppm
Jun 19 02:42:13 dist-test-slave-zpbg krb5kdc[19797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836933, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:42:13.743779 19810 init.cc:377] Logged in from keytab as kudu/127.5.241.126@KRBTEST.COM (short username kudu)
I20260619 02:42:13.744251 19810 webserver.cc:460] Webserver started at http://127.5.241.126:37257/ using document root <none> and password file <none>
I20260619 02:42:13.744485 19810 fs_manager.cc:359] Metadata directory not provided
I20260619 02:42:13.744554 19810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:42:13.744688 19810 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:42:13.745756 19810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/master-0/data/instance:
uuid: "ff74bd03ec4f43e98fb7e337f818650c"
format_stamp: "Formatted at 2026-06-19 02:42:13 on dist-test-slave-zpbg"
server_key: "80fc0bb3e5cc176318063434d81fe8b9"
server_key_iv: "6386c7c0580abf292387e5fa31869cb9"
server_key_version: "encryptionkey@0"
I20260619 02:42:13.746125 19810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/master-0/wal/instance:
uuid: "ff74bd03ec4f43e98fb7e337f818650c"
format_stamp: "Formatted at 2026-06-19 02:42:13 on dist-test-slave-zpbg"
server_key: "80fc0bb3e5cc176318063434d81fe8b9"
server_key_iv: "6386c7c0580abf292387e5fa31869cb9"
server_key_version: "encryptionkey@0"
I20260619 02:42:13.747610 19810 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:42:13.748406 19825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:13.748600 19810 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:42:13.748703 19810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "ff74bd03ec4f43e98fb7e337f818650c"
format_stamp: "Formatted at 2026-06-19 02:42:13 on dist-test-slave-zpbg"
server_key: "80fc0bb3e5cc176318063434d81fe8b9"
server_key_iv: "6386c7c0580abf292387e5fa31869cb9"
server_key_version: "encryptionkey@0"
I20260619 02:42:13.748808 19810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:42:13.762844 19810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:42:13.765102 19810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:42:13.765275 19810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:42:13.769536 19810 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.126:45629
I20260619 02:42:13.769583 19877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.126:45629 every 8 connection(s)
I20260619 02:42:13.769958 19810 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:42:13.771062 19878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:42:13.773376 19878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c: Bootstrap starting.
I20260619 02:42:13.774120 19878 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c: Neither blocks nor log segments found. Creating new log.
I20260619 02:42:13.774358 19878 log.cc:824] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c: Log is configured to *not* fsync() on all Append() calls
I20260619 02:42:13.775105 19878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c: No bootstrap required, opened a new log
I20260619 02:42:13.776376 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 19810
I20260619 02:42:13.776489 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/master-0/wal/instance
I20260619 02:42:13.776679 6085 external_mini_cluster.cc:1428] Setting key aad62199cfe63d49322c1e1ef235c293
I20260619 02:42:13.776974 19878 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff74bd03ec4f43e98fb7e337f818650c" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45629 } }
I20260619 02:42:13.777158 19878 raft_consensus.cc:374] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:42:13.777202 19878 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff74bd03ec4f43e98fb7e337f818650c, State: Initialized, Role: FOLLOWER
I20260619 02:42:13.777324 19878 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff74bd03ec4f43e98fb7e337f818650c" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45629 } }
I20260619 02:42:13.777420 19878 raft_consensus.cc:388] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:42:13.777465 19878 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:42:13.777520 19878 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:42:13.778359 19878 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff74bd03ec4f43e98fb7e337f818650c" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45629 } }
I20260619 02:42:13.778481 19878 leader_election.cc:302] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ff74bd03ec4f43e98fb7e337f818650c; no voters:
I20260619 02:42:13.778636 19878 leader_election.cc:288] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:42:13.778666 19883 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:42:13.778817 19883 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c [term 1 LEADER]: Becoming Leader. State: Replica: ff74bd03ec4f43e98fb7e337f818650c, State: Running, Role: LEADER
I20260619 02:42:13.778913 19878 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:42:13.778952 19883 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff74bd03ec4f43e98fb7e337f818650c" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45629 } }
I20260619 02:42:13.779328 19883 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff74bd03ec4f43e98fb7e337f818650c. Latest consensus state: current_term: 1 leader_uuid: "ff74bd03ec4f43e98fb7e337f818650c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff74bd03ec4f43e98fb7e337f818650c" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45629 } } }
I20260619 02:42:13.779342 19884 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ff74bd03ec4f43e98fb7e337f818650c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff74bd03ec4f43e98fb7e337f818650c" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45629 } } }
I20260619 02:42:13.779486 19883 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c [sys.catalog]: This master's current role is: LEADER
I20260619 02:42:13.779500 19884 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c [sys.catalog]: This master's current role is: LEADER
I20260619 02:42:13.779826 19892 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:42:13.780324 19892 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:42:13.782179 19892 catalog_manager.cc:1347] Generated new cluster ID: 535ce5b6d51544439284599f7eea6080
I20260619 02:42:13.782233 19892 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 02:42:13 dist-test-slave-zpbg krb5kdc[19797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:42:13.788403 19882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:13.777231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45602 (local address 127.5.241.126:45629)
0619 02:42:13.777444 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:13.777453 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:13.777487 (+ 34us) server_negotiation.cc:407] Connection header received
0619 02:42:13.777641 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:13.777660 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:13.777705 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:13.777841 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:42:13.778470 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:13.779907 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:13.781011 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:13.781490 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:13.781624 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:13.784408 (+ 2784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:42:13.784412 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:42:13.784412 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:42:13.784424 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:42:13.786408 (+ 1984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:13.786811 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:13.786814 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:13.786814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:13.786929 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:13.787134 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:13.787135 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:13.787135 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:13.787440 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:42:13.787987 (+ 547us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:13.788061 (+ 74us) server_negotiation.cc:299] Negotiation successful
0619 02:42:13.788198 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":95,"threads_started":1}
I20260619 02:42:13.809916 19892 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:42:13.810685 19892 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:42:13.816792 19892 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c: Generated new TSK 0
I20260619 02:42:13.816977 19892 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.241.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab.
Entry for principal kudu/127.5.241.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab.
I20260619 02:42:13.849978 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.65:0
--local_ip_for_outbound_sockets=127.5.241.65
--webserver_interface=127.5.241.65
--webserver_port=0
--tserver_master_addrs=127.5.241.126:45629
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab
--principal=kudu/127.5.241.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:42191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:42:13.932125 19904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:42:13.932329 19904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:42:13.932353 19904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:42:13.933508 19904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:42:13.933554 19904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:42:13.933580 19904 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.65
I20260619 02:42:13.935145 19904 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:42191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab
--principal=kudu/127.5.241.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.241.65
--webserver_port=0
--tserver_master_addrs=127.5.241.126:45629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19904
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.65
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:42:13.935402 19904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:42:13.935639 19904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:42:13.940735 19909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:42:13.940819 19904 server_base.cc:1034] running on GCE node
W20260619 02:42:13.940732 19912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:13.940726 19910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:42:13.941180 19904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:42:13.941403 19904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:42:13.942548 19904 hybrid_clock.cc:648] HybridClock initialized: now 1781836933942524 us; error 44 us; skew 500 ppm
Jun 19 02:42:13 dist-test-slave-zpbg krb5kdc[19797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836933, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:42:13.945273 19904 init.cc:377] Logged in from keytab as kudu/127.5.241.65@KRBTEST.COM (short username kudu)
I20260619 02:42:13.945713 19904 webserver.cc:460] Webserver started at http://127.5.241.65:35775/ using document root <none> and password file <none>
I20260619 02:42:13.945963 19904 fs_manager.cc:359] Metadata directory not provided
I20260619 02:42:13.946009 19904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:42:13.946138 19904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:42:13.947284 19904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-0/data/instance:
uuid: "00c9712fe62f4d23a46d6270a86101e9"
format_stamp: "Formatted at 2026-06-19 02:42:13 on dist-test-slave-zpbg"
server_key: "f20602628e991cb38fd1cd4c336c232f"
server_key_iv: "44fddaa384111960f22326a1fa79900d"
server_key_version: "encryptionkey@0"
I20260619 02:42:13.947592 19904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-0/wal/instance:
uuid: "00c9712fe62f4d23a46d6270a86101e9"
format_stamp: "Formatted at 2026-06-19 02:42:13 on dist-test-slave-zpbg"
server_key: "f20602628e991cb38fd1cd4c336c232f"
server_key_iv: "44fddaa384111960f22326a1fa79900d"
server_key_version: "encryptionkey@0"
I20260619 02:42:13.948930 19904 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:42:13.949692 19919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:13.949873 19904 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:42:13.949955 19904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-0/wal
uuid: "00c9712fe62f4d23a46d6270a86101e9"
format_stamp: "Formatted at 2026-06-19 02:42:13 on dist-test-slave-zpbg"
server_key: "f20602628e991cb38fd1cd4c336c232f"
server_key_iv: "44fddaa384111960f22326a1fa79900d"
server_key_version: "encryptionkey@0"
I20260619 02:42:13.950009 19904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:42:13.965157 19904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:42:13.967530 19904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:42:13.967643 19904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:42:13.967825 19904 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:42:13.968140 19904 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:42:13.968173 19904 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:13.968235 19904 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:42:13.968261 19904 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:13.973901 19904 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.65:46275
I20260619 02:42:13.973985 20032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.65:46275 every 8 connection(s)
I20260619 02:42:13.974325 19904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
I20260619 02:42:13.974962 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 19904
I20260619 02:42:13.975071 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-0/wal/instance
I20260619 02:42:13.975308 6085 external_mini_cluster.cc:1428] Setting key d82c2848a4b33699a5fbe76619460905
Jun 19 02:42:13 dist-test-slave-zpbg krb5kdc[19797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836933, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.65@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.241.66@KRBTEST.COM; defaulting to no policy
I20260619 02:42:13.985258 19882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:13.975524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:53011 (local address 127.5.241.126:45629)
0619 02:42:13.975635 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:13.975638 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:13.975649 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:42:13.976231 (+ 582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:13.976232 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:13.976245 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:13.976331 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:42:13.977260 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:13.977875 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:13.979468 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:13.979721 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:13.979784 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:13.982271 (+ 2487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:42:13.982273 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:42:13.982273 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:42:13.982281 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:42:13.983716 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:13.984170 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:13.984172 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:13.984172 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:13.984232 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:13.984497 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:13.984498 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:13.984498 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:13.984615 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:42:13.984694 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:13.985033 (+ 339us) server_negotiation.cc:299] Negotiation successful
0619 02:42:13.985159 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260619 02:42:13.985415 20035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:13.975219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.65:53011)
0619 02:42:13.975436 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:13.975507 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:13.976052 (+ 545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:13.976388 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:13.976392 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:13.976965 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:42:13.977161 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:13.977165 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:13.977971 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:13.977973 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:13.979379 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:13.979382 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:13.979812 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:13.979813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:13.979975 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:13.980513 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:42:13.980524 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:42:13.982171 (+ 1647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:42:13.983821 (+ 1650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:13.983829 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:13.983831 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:13.984083 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:13.984307 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:13.984308 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:13.984308 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:13.984425 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:13.984684 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:42:13.984691 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:42:13.984884 (+ 193us) client_negotiation.cc:769] Sending connection context
0619 02:42:13.985000 (+ 116us) client_negotiation.cc:240] Negotiation successful
0619 02:42:13.985178 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":99,"threads_started":1}
I20260619 02:42:13.985908 20033 heartbeater.cc:347] Connected to a master server at 127.5.241.126:45629
I20260619 02:42:13.985999 20033 heartbeater.cc:464] Registering TS with master...
I20260619 02:42:13.986223 20033 heartbeater.cc:511] Master 127.5.241.126:45629 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.66@KRBTEST.COM" created.
I20260619 02:42:13.986817 19842 ts_manager.cc:194] Registered new tserver with Master: 00c9712fe62f4d23a46d6270a86101e9 (127.5.241.65:46275)
I20260619 02:42:13.987421 19842 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.65@KRBTEST.COM'} at 127.5.241.65:53011
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab.
Entry for principal kudu/127.5.241.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab.
I20260619 02:42:14.002344 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.66:0
--local_ip_for_outbound_sockets=127.5.241.66
--webserver_interface=127.5.241.66
--webserver_port=0
--tserver_master_addrs=127.5.241.126:45629
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab
--principal=kudu/127.5.241.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:42191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:42:14.084019 20038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:42:14.084219 20038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:42:14.084244 20038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:42:14.085314 20038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:42:14.085347 20038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:42:14.085373 20038 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.66
I20260619 02:42:14.086822 20038 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:42191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab
--principal=kudu/127.5.241.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.241.66
--webserver_port=0
--tserver_master_addrs=127.5.241.126:45629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20038
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.66
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:42:14.087003 20038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:42:14.087172 20038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:42:14.092202 20046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:14.092203 20044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:14.092272 20043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:42:14.092559 20038 server_base.cc:1034] running on GCE node
I20260619 02:42:14.092783 20038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:42:14.092978 20038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:42:14.094111 20038 hybrid_clock.cc:648] HybridClock initialized: now 1781836934094100 us; error 28 us; skew 500 ppm
Jun 19 02:42:14 dist-test-slave-zpbg krb5kdc[19797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836934, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:42:14.096568 20038 init.cc:377] Logged in from keytab as kudu/127.5.241.66@KRBTEST.COM (short username kudu)
I20260619 02:42:14.096999 20038 webserver.cc:460] Webserver started at http://127.5.241.66:33589/ using document root <none> and password file <none>
I20260619 02:42:14.097218 20038 fs_manager.cc:359] Metadata directory not provided
I20260619 02:42:14.097285 20038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:42:14.097434 20038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:42:14.098453 20038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-1/data/instance:
uuid: "816a43ca3f4842079542408ac9fde61f"
format_stamp: "Formatted at 2026-06-19 02:42:14 on dist-test-slave-zpbg"
server_key: "7d7d61fda9659281d6367e7697c06795"
server_key_iv: "903a2433cb5cf122f717fea700121717"
server_key_version: "encryptionkey@0"
I20260619 02:42:14.098881 20038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-1/wal/instance:
uuid: "816a43ca3f4842079542408ac9fde61f"
format_stamp: "Formatted at 2026-06-19 02:42:14 on dist-test-slave-zpbg"
server_key: "7d7d61fda9659281d6367e7697c06795"
server_key_iv: "903a2433cb5cf122f717fea700121717"
server_key_version: "encryptionkey@0"
I20260619 02:42:14.100164 20038 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:42:14.100870 20053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:14.101048 20038 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:42:14.101145 20038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-1/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-1/wal
uuid: "816a43ca3f4842079542408ac9fde61f"
format_stamp: "Formatted at 2026-06-19 02:42:14 on dist-test-slave-zpbg"
server_key: "7d7d61fda9659281d6367e7697c06795"
server_key_iv: "903a2433cb5cf122f717fea700121717"
server_key_version: "encryptionkey@0"
I20260619 02:42:14.101225 20038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:42:14.124856 20038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:42:14.127440 20038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:42:14.127564 20038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:42:14.127765 20038 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:42:14.128095 20038 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:42:14.128129 20038 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:14.128187 20038 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:42:14.128226 20038 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:14.133922 20038 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.66:46401
I20260619 02:42:14.134101 20166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.66:46401 every 8 connection(s)
I20260619 02:42:14.134346 20038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
I20260619 02:42:14.137238 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 20038
I20260619 02:42:14.137312 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-1/wal/instance
I20260619 02:42:14.137501 6085 external_mini_cluster.cc:1428] Setting key 57574bd7834fb8abfc1c545cbdea4dbf
Jun 19 02:42:14 dist-test-slave-zpbg krb5kdc[19797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836934, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.66@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:42:14.144549 19882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:14.135119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:46159 (local address 127.5.241.126:45629)
0619 02:42:14.135195 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:14.135197 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:14.135381 (+ 184us) server_negotiation.cc:407] Connection header received
0619 02:42:14.135877 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:14.135879 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:14.135918 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:14.135998 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:42:14.136997 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:14.137738 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:14.138879 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:14.139098 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:14.139183 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:14.141576 (+ 2393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:42:14.141577 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:42:14.141578 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:42:14.141586 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:42:14.143096 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:14.143548 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:14.143550 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:14.143550 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:14.143610 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:14.143862 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:14.143863 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:14.143863 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:14.143979 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:42:14.144071 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:14.144370 (+ 299us) server_negotiation.cc:299] Negotiation successful
0619 02:42:14.144459 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 02:42:14.144757 20169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:14.135069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.66:46159)
0619 02:42:14.135245 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:14.135287 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:14.135767 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:14.136068 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:14.136072 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:14.136652 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:42:14.136842 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:14.136847 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:14.137855 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:14.137856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:14.138748 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:14.138751 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:14.139192 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:14.139193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:14.139344 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:14.139892 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:42:14.139903 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:42:14.141477 (+ 1574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:42:14.143219 (+ 1742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:14.143226 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:14.143228 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:14.143455 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:14.143680 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:14.143681 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:14.143681 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:14.143792 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:14.144061 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:42:14.144067 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:42:14.144241 (+ 174us) client_negotiation.cc:769] Sending connection context
0619 02:42:14.144349 (+ 108us) client_negotiation.cc:240] Negotiation successful
0619 02:42:14.144507 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":100,"threads_started":1}
I20260619 02:42:14.145576 20167 heartbeater.cc:347] Connected to a master server at 127.5.241.126:45629
I20260619 02:42:14.145678 20167 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.5.241.67@KRBTEST.COM; defaulting to no policy
I20260619 02:42:14.145890 20167 heartbeater.cc:511] Master 127.5.241.126:45629 requested a full tablet report, sending...
I20260619 02:42:14.146389 19842 ts_manager.cc:194] Registered new tserver with Master: 816a43ca3f4842079542408ac9fde61f (127.5.241.66:46401)
I20260619 02:42:14.146888 19842 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.66@KRBTEST.COM'} at 127.5.241.66:46159
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab.
Entry for principal kudu/127.5.241.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab.
I20260619 02:42:14.164234 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.67:0
--local_ip_for_outbound_sockets=127.5.241.67
--webserver_interface=127.5.241.67
--webserver_port=0
--tserver_master_addrs=127.5.241.126:45629
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab
--principal=kudu/127.5.241.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:42191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:42:14.243278 20172 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:42:14.243477 20172 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:42:14.243500 20172 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:42:14.244761 20172 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:42:14.244818 20172 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:42:14.244846 20172 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.67
I20260619 02:42:14.246240 20172 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:42191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab
--principal=kudu/127.5.241.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.241.67
--webserver_port=0
--tserver_master_addrs=127.5.241.126:45629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20172
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.67
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:42:14.246425 20172 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:42:14.246595 20172 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:42:14.251525 20180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:14.251526 20177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:42:14.251861 20172 server_base.cc:1034] running on GCE node
W20260619 02:42:14.251705 20178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:42:14.252130 20172 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:42:14.252432 20172 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:42:14.253619 20172 hybrid_clock.cc:648] HybridClock initialized: now 1781836934253608 us; error 27 us; skew 500 ppm
Jun 19 02:42:14 dist-test-slave-zpbg krb5kdc[19797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836934, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:42:14.256155 20172 init.cc:377] Logged in from keytab as kudu/127.5.241.67@KRBTEST.COM (short username kudu)
I20260619 02:42:14.256585 20172 webserver.cc:460] Webserver started at http://127.5.241.67:35069/ using document root <none> and password file <none>
I20260619 02:42:14.256824 20172 fs_manager.cc:359] Metadata directory not provided
I20260619 02:42:14.256876 20172 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:42:14.257048 20172 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:42:14.258123 20172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-2/data/instance:
uuid: "ca02d260fb4842dcb4121f3c88a64508"
format_stamp: "Formatted at 2026-06-19 02:42:14 on dist-test-slave-zpbg"
server_key: "bf1afc2bede4ed86072d0561befb329f"
server_key_iv: "c0dab91368041e3d6923351a0eaf8cc1"
server_key_version: "encryptionkey@0"
I20260619 02:42:14.258490 20172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-2/wal/instance:
uuid: "ca02d260fb4842dcb4121f3c88a64508"
format_stamp: "Formatted at 2026-06-19 02:42:14 on dist-test-slave-zpbg"
server_key: "bf1afc2bede4ed86072d0561befb329f"
server_key_iv: "c0dab91368041e3d6923351a0eaf8cc1"
server_key_version: "encryptionkey@0"
I20260619 02:42:14.259835 20172 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:42:14.260641 20187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:14.260835 20172 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:42:14.260949 20172 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-2/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-2/wal
uuid: "ca02d260fb4842dcb4121f3c88a64508"
format_stamp: "Formatted at 2026-06-19 02:42:14 on dist-test-slave-zpbg"
server_key: "bf1afc2bede4ed86072d0561befb329f"
server_key_iv: "c0dab91368041e3d6923351a0eaf8cc1"
server_key_version: "encryptionkey@0"
I20260619 02:42:14.261037 20172 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:42:14.276710 20172 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:42:14.278932 20172 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:42:14.279083 20172 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:42:14.279304 20172 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:42:14.279664 20172 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:42:14.279716 20172 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:14.279758 20172 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:42:14.279786 20172 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:14.285971 20172 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.67:39503
I20260619 02:42:14.286011 20300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.67:39503 every 8 connection(s)
I20260619 02:42:14.286347 20172 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
I20260619 02:42:14.288872 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 20172
I20260619 02:42:14.288981 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-2/wal/instance
I20260619 02:42:14.289152 6085 external_mini_cluster.cc:1428] Setting key 9530d601c7cec7ac2d072f4b94d118b5
Jun 19 02:42:14 dist-test-slave-zpbg krb5kdc[19797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836934, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.67@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:42:14.295997 19882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:14.287183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:51385 (local address 127.5.241.126:45629)
0619 02:42:14.287227 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:14.287229 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:14.287458 (+ 229us) server_negotiation.cc:407] Connection header received
0619 02:42:14.287984 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:14.287986 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:14.288001 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:14.288092 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:42:14.289002 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:14.289734 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:14.290888 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:14.291084 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:14.291149 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:14.293293 (+ 2144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:42:14.293294 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:42:14.293294 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:42:14.293301 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:42:14.294501 (+ 1200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:14.294979 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:14.294980 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:14.294980 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:14.295038 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:14.295352 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:14.295353 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:14.295354 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:14.295485 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:42:14.295562 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:14.295861 (+ 299us) server_negotiation.cc:299] Negotiation successful
0619 02:42:14.295932 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 02:42:14.296217 20303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:14.287157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.67:51385)
0619 02:42:14.287323 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:14.287374 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:14.287877 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:14.288141 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:14.288145 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:14.288701 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:42:14.288898 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:14.288903 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:14.289820 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:14.289821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:14.290720 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:14.290723 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:14.291166 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:14.291167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:14.291284 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:14.291853 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:42:14.291862 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:42:14.293194 (+ 1332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:42:14.294584 (+ 1390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:14.294590 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:14.294591 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:14.294905 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:14.295135 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:14.295136 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:14.295136 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:14.295255 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:14.295565 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:42:14.295570 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:42:14.295718 (+ 148us) client_negotiation.cc:769] Sending connection context
0619 02:42:14.295827 (+ 109us) client_negotiation.cc:240] Negotiation successful
0619 02:42:14.295956 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":100,"threads_started":1}
I20260619 02:42:14.296759 20301 heartbeater.cc:347] Connected to a master server at 127.5.241.126:45629
I20260619 02:42:14.296851 20301 heartbeater.cc:464] Registering TS with master...
I20260619 02:42:14.297008 20301 heartbeater.cc:511] Master 127.5.241.126:45629 requested a full tablet report, sending...
I20260619 02:42:14.297405 19842 ts_manager.cc:194] Registered new tserver with Master: ca02d260fb4842dcb4121f3c88a64508 (127.5.241.67:39503)
I20260619 02:42:14.297761 19842 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.67@KRBTEST.COM'} at 127.5.241.67:51385
I20260619 02:42:14.300918 6085 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:42:14.308727 19882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:14.302253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45610 (local address 127.5.241.126:45629)
0619 02:42:14.302445 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:14.302448 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:14.302538 (+ 90us) server_negotiation.cc:407] Connection header received
0619 02:42:14.302630 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:14.302631 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:14.302641 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:14.302700 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:42:14.303311 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:14.303875 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:14.304591 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:14.304803 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:14.304888 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:14.306152 (+ 1264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:42:14.306154 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:42:14.306154 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:42:14.306162 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:42:14.307466 (+ 1304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:14.307869 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:14.307870 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:14.307870 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:14.307932 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:14.308131 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:14.308132 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:14.308133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:14.308297 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:42:14.308404 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:14.308544 (+ 140us) server_negotiation.cc:299] Negotiation successful
0619 02:42:14.308625 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161}
I20260619 02:42:14.310303 19842 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45610:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:42:14.312027 19842 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:42:14.318543 20309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:14.314221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.66:46401 (local address 127.0.0.1:34220)
0619 02:42:14.314558 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:14.314565 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:14.314620 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:14.314961 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:14.314964 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:14.314967 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:42:14.315032 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:14.315034 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:14.316087 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:14.316088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:14.317242 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:14.317245 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:14.318093 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:14.318093 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:14.318172 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:14.318185 (+ 13us) client_negotiation.cc:769] Sending connection context
0619 02:42:14.318399 (+ 214us) client_negotiation.cc:240] Negotiation successful
0619 02:42:14.318470 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"mutex_wait_us":1,"thread_start_us":56,"threads_started":1}
I20260619 02:42:14.318553 20312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:14.314488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34220 (local address 127.5.241.66:46401)
0619 02:42:14.314672 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:14.314677 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:14.314687 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:42:14.314732 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:14.314734 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:14.314746 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:14.314928 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:42:14.315097 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:14.315980 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:14.317315 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:14.318008 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:14.318092 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:14.318154 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:14.318418 (+ 264us) server_negotiation.cc:299] Negotiation successful
0619 02:42:14.318467 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":77,"threads_started":1}
I20260619 02:42:14.318807 20311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:14.314221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.67:39503 (local address 127.0.0.1:41410)
0619 02:42:14.314384 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:14.314422 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:14.314481 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:14.315378 (+ 897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:14.315379 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:14.315381 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:42:14.315422 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:14.315423 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:14.316565 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:14.316566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:14.317706 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:14.317709 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:14.318589 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:14.318590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:14.318649 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:14.318652 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:42:14.318680 (+ 28us) client_negotiation.cc:240] Negotiation successful
0619 02:42:14.318718 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":1,"thread_start_us":66,"threads_started":1}
I20260619 02:42:14.319169 20314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:14.314671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46128 (local address 127.5.241.65:46275)
0619 02:42:14.314873 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:14.314877 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:14.314887 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:42:14.314926 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:14.314928 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:14.314943 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:14.315055 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:42:14.315267 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:14.316400 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:14.317623 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:14.318358 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:14.318431 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:14.318486 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:14.319035 (+ 549us) server_negotiation.cc:299] Negotiation successful
0619 02:42:14.319085 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":87,"threads_started":1}
I20260619 02:42:14.319197 20313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:14.314472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41410 (local address 127.5.241.67:39503)
0619 02:42:14.315197 (+ 725us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:14.315200 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:14.315207 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:42:14.315267 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:14.315268 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:14.315284 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:14.315367 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:42:14.315480 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:14.316483 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:14.317803 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:14.318502 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:14.318980 (+ 478us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:14.319050 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:14.319075 (+ 25us) server_negotiation.cc:299] Negotiation successful
0619 02:42:14.319117 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":696,"thread_start_us":97,"threads_started":1}
I20260619 02:42:14.319574 20101 tablet_service.cc:1467] Processing CreateTablet for tablet 0f45bf4a084a4ee790312b3144923be5 (DEFAULT_TABLE table=test-table [id=1642464714644277a03da04d490764ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:42:14.319708 20235 tablet_service.cc:1467] Processing CreateTablet for tablet 0f45bf4a084a4ee790312b3144923be5 (DEFAULT_TABLE table=test-table [id=1642464714644277a03da04d490764ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:42:14.319955 20101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f45bf4a084a4ee790312b3144923be5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:42:14.319972 20235 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f45bf4a084a4ee790312b3144923be5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:42:14.323390 20315 tablet_bootstrap.cc:492] T 0f45bf4a084a4ee790312b3144923be5 P ca02d260fb4842dcb4121f3c88a64508: Bootstrap starting.
I20260619 02:42:14.319099 20310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:14.314233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:46275 (local address 127.0.0.1:46128)
0619 02:42:14.314654 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:14.314666 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:14.314740 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:14.315138 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:14.315139 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:14.315141 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:42:14.315185 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:14.315187 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:14.316506 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:14.316507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:14.317530 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:14.317532 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:14.318524 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:14.318526 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:14.318604 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:14.318608 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:42:14.318995 (+ 387us) client_negotiation.cc:240] Negotiation successful
0619 02:42:14.319037 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":407,"mutex_wait_us":25,"thread_start_us":53,"threads_started":1}
I20260619 02:42:14.323601 20316 tablet_bootstrap.cc:492] T 0f45bf4a084a4ee790312b3144923be5 P 816a43ca3f4842079542408ac9fde61f: Bootstrap starting.
I20260619 02:42:14.324172 20315 tablet_bootstrap.cc:654] T 0f45bf4a084a4ee790312b3144923be5 P ca02d260fb4842dcb4121f3c88a64508: Neither blocks nor log segments found. Creating new log.
I20260619 02:42:14.324208 20316 tablet_bootstrap.cc:654] T 0f45bf4a084a4ee790312b3144923be5 P 816a43ca3f4842079542408ac9fde61f: Neither blocks nor log segments found. Creating new log.
I20260619 02:42:14.324213 19967 tablet_service.cc:1467] Processing CreateTablet for tablet 0f45bf4a084a4ee790312b3144923be5 (DEFAULT_TABLE table=test-table [id=1642464714644277a03da04d490764ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:42:14.324580 19967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f45bf4a084a4ee790312b3144923be5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:42:14.324807 20315 log.cc:824] T 0f45bf4a084a4ee790312b3144923be5 P ca02d260fb4842dcb4121f3c88a64508: Log is configured to *not* fsync() on all Append() calls
I20260619 02:42:14.324808 20316 log.cc:824] T 0f45bf4a084a4ee790312b3144923be5 P 816a43ca3f4842079542408ac9fde61f: Log is configured to *not* fsync() on all Append() calls
I20260619 02:42:14.325593 20315 tablet_bootstrap.cc:492] T 0f45bf4a084a4ee790312b3144923be5 P ca02d260fb4842dcb4121f3c88a64508: No bootstrap required, opened a new log
I20260619 02:42:14.325661 20315 ts_tablet_manager.cc:1397] T 0f45bf4a084a4ee790312b3144923be5 P ca02d260fb4842dcb4121f3c88a64508: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:42:14.325963 20316 tablet_bootstrap.cc:492] T 0f45bf4a084a4ee790312b3144923be5 P 816a43ca3f4842079542408ac9fde61f: No bootstrap required, opened a new log
I20260619 02:42:14.326031 20316 ts_tablet_manager.cc:1397] T 0f45bf4a084a4ee790312b3144923be5 P 816a43ca3f4842079542408ac9fde61f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 02:42:14.327698 20319 tablet_bootstrap.cc:492] T 0f45bf4a084a4ee790312b3144923be5 P 00c9712fe62f4d23a46d6270a86101e9: Bootstrap starting.
I20260619 02:42:14.327666 20315 raft_consensus.cc:348] T 0f45bf4a084a4ee790312b3144923be5 P ca02d260fb4842dcb4121f3c88a64508 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca02d260fb4842dcb4121f3c88a64508" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 39503 } } peers { permanent_uuid: "816a43ca3f4842079542408ac9fde61f" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 46401 } } peers { permanent_uuid: "00c9712fe62f4d23a46d6270a86101e9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46275 } }
I20260619 02:42:14.327855 20315 raft_consensus.cc:374] T 0f45bf4a084a4ee790312b3144923be5 P ca02d260fb4842dcb4121f3c88a64508 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:42:14.327901 20315 raft_consensus.cc:729] T 0f45bf4a084a4ee790312b3144923be5 P ca02d260fb4842dcb4121f3c88a64508 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca02d260fb4842dcb4121f3c88a64508, State: Initialized, Role: FOLLOWER
I20260619 02:42:14.328028 20315 consensus_queue.cc:260] T 0f45bf4a084a4ee790312b3144923be5 P ca02d260fb4842dcb4121f3c88a64508 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca02d260fb4842dcb4121f3c88a64508" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 39503 } } peers { permanent_uuid: "816a43ca3f4842079542408ac9fde61f" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 46401 } } peers { permanent_uuid: "00c9712fe62f4d23a46d6270a86101e9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46275 } }
I20260619 02:42:14.327965 20316 raft_consensus.cc:348] T 0f45bf4a084a4ee790312b3144923be5 P 816a43ca3f4842079542408ac9fde61f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca02d260fb4842dcb4121f3c88a64508" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 39503 } } peers { permanent_uuid: "816a43ca3f4842079542408ac9fde61f" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 46401 } } peers { permanent_uuid: "00c9712fe62f4d23a46d6270a86101e9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46275 } }
I20260619 02:42:14.328172 20316 raft_consensus.cc:374] T 0f45bf4a084a4ee790312b3144923be5 P 816a43ca3f4842079542408ac9fde61f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:42:14.328212 20316 raft_consensus.cc:729] T 0f45bf4a084a4ee790312b3144923be5 P 816a43ca3f4842079542408ac9fde61f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 816a43ca3f4842079542408ac9fde61f, State: Initialized, Role: FOLLOWER
I20260619 02:42:14.328320 20315 ts_tablet_manager.cc:1428] T 0f45bf4a084a4ee790312b3144923be5 P ca02d260fb4842dcb4121f3c88a64508: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 02:42:14.328320 20316 consensus_queue.cc:260] T 0f45bf4a084a4ee790312b3144923be5 P 816a43ca3f4842079542408ac9fde61f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca02d260fb4842dcb4121f3c88a64508" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 39503 } } peers { permanent_uuid: "816a43ca3f4842079542408ac9fde61f" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 46401 } } peers { permanent_uuid: "00c9712fe62f4d23a46d6270a86101e9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46275 } }
I20260619 02:42:14.328451 20319 tablet_bootstrap.cc:654] T 0f45bf4a084a4ee790312b3144923be5 P 00c9712fe62f4d23a46d6270a86101e9: Neither blocks nor log segments found. Creating new log.
I20260619 02:42:14.328444 20301 heartbeater.cc:503] Master 127.5.241.126:45629 was elected leader, sending a full tablet report...
I20260619 02:42:14.328614 20167 heartbeater.cc:503] Master 127.5.241.126:45629 was elected leader, sending a full tablet report...
I20260619 02:42:14.328620 20316 ts_tablet_manager.cc:1428] T 0f45bf4a084a4ee790312b3144923be5 P 816a43ca3f4842079542408ac9fde61f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 02:42:14.329054 20319 log.cc:824] T 0f45bf4a084a4ee790312b3144923be5 P 00c9712fe62f4d23a46d6270a86101e9: Log is configured to *not* fsync() on all Append() calls
I20260619 02:42:14.329924 20319 tablet_bootstrap.cc:492] T 0f45bf4a084a4ee790312b3144923be5 P 00c9712fe62f4d23a46d6270a86101e9: No bootstrap required, opened a new log
I20260619 02:42:14.329998 20319 ts_tablet_manager.cc:1397] T 0f45bf4a084a4ee790312b3144923be5 P 00c9712fe62f4d23a46d6270a86101e9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:42:14.331365 20319 raft_consensus.cc:348] T 0f45bf4a084a4ee790312b3144923be5 P 00c9712fe62f4d23a46d6270a86101e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca02d260fb4842dcb4121f3c88a64508" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 39503 } } peers { permanent_uuid: "816a43ca3f4842079542408ac9fde61f" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 46401 } } peers { permanent_uuid: "00c9712fe62f4d23a46d6270a86101e9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46275 } }
I20260619 02:42:14.331492 20319 raft_consensus.cc:374] T 0f45bf4a084a4ee790312b3144923be5 P 00c9712fe62f4d23a46d6270a86101e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:42:14.331514 20319 raft_consensus.cc:729] T 0f45bf4a084a4ee790312b3144923be5 P 00c9712fe62f4d23a46d6270a86101e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00c9712fe62f4d23a46d6270a86101e9, State: Initialized, Role: FOLLOWER
I20260619 02:42:14.331621 20319 consensus_queue.cc:260] T 0f45bf4a084a4ee790312b3144923be5 P 00c9712fe62f4d23a46d6270a86101e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca02d260fb4842dcb4121f3c88a64508" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 39503 } } peers { permanent_uuid: "816a43ca3f4842079542408ac9fde61f" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 46401 } } peers { permanent_uuid: "00c9712fe62f4d23a46d6270a86101e9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46275 } }
I20260619 02:42:14.331861 20319 ts_tablet_manager.cc:1428] T 0f45bf4a084a4ee790312b3144923be5 P 00c9712fe62f4d23a46d6270a86101e9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:42:14.331924 20033 heartbeater.cc:503] Master 127.5.241.126:45629 was elected leader, sending a full tablet report...
W20260619 02:42:14.384925 20168 tablet.cc:2369] T 0f45bf4a084a4ee790312b3144923be5 P 816a43ca3f4842079542408ac9fde61f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:42:14.395381 20323 raft_consensus.cc:482] T 0f45bf4a084a4ee790312b3144923be5 P 00c9712fe62f4d23a46d6270a86101e9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:42:14.395525 20323 raft_consensus.cc:504] T 0f45bf4a084a4ee790312b3144923be5 P 00c9712fe62f4d23a46d6270a86101e9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca02d260fb4842dcb4121f3c88a64508" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 39503 } } peers { permanent_uuid: "816a43ca3f4842079542408ac9fde61f" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 46401 } } peers { permanent_uuid: "00c9712fe62f4d23a46d6270a86101e9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46275 } }
I20260619 02:42:14.395964 20323 leader_election.cc:288] T 0f45bf4a084a4ee790312b3144923be5 P 00c9712fe62f4d23a46d6270a86101e9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca02d260fb4842dcb4121f3c88a64508 (127.5.241.67:39503), 816a43ca3f4842079542408ac9fde61f (127.5.241.66:46401)
I20260619 02:42:14.400151 20313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:14.396279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:53917 (local address 127.5.241.67:39503)
0619 02:42:14.396327 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:14.396329 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:14.396443 (+ 114us) server_negotiation.cc:407] Connection header received
0619 02:42:14.396476 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:14.396478 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:14.396488 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:14.396540 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:42:14.396747 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:14.397480 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:14.399113 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:14.399757 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:14.399843 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:14.399852 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:14.400038 (+ 186us) server_negotiation.cc:299] Negotiation successful
0619 02:42:14.400078 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 02:42:14.400151 20312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:14.396281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:35451 (local address 127.5.241.66:46401)
0619 02:42:14.396327 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:14.396329 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:14.396340 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:42:14.396408 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:14.396408 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:14.396437 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:14.396507 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:42:14.396722 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:14.397480 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:14.399180 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:14.399755 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:14.399843 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:14.399852 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:14.400038 (+ 186us) server_negotiation.cc:299] Negotiation successful
0619 02:42:14.400078 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 02:42:14.400166 20035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:14.396158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.66:46401 (local address 127.5.241.65:35451)
0619 02:42:14.396252 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:14.396282 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:14.396334 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:14.396550 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:14.396552 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:14.396554 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:42:14.396629 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:14.396632 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:14.397594 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:14.397596 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:14.398995 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:14.398998 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:14.399859 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:14.399860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:14.399952 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:14.399953 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 02:42:14.400016 (+ 63us) client_negotiation.cc:240] Negotiation successful
0619 02:42:14.400067 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66,"mutex_wait_us":78}
I20260619 02:42:14.400172 20324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:14.396169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.67:39503 (local address 127.5.241.65:53917)
0619 02:42:14.396368 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:14.396377 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:14.396429 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:14.396566 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:14.396566 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:14.396568 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:42:14.396646 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:14.396647 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:14.397594 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:14.397596 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:14.398995 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:14.398998 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:14.399861 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:14.399862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:14.399946 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:14.399951 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 02:42:14.400024 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 02:42:14.400067 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":81,"thread_start_us":129,"threads_started":1}
I20260619 02:42:14.400574 20255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f45bf4a084a4ee790312b3144923be5" candidate_uuid: "00c9712fe62f4d23a46d6270a86101e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca02d260fb4842dcb4121f3c88a64508" is_pre_election: true
I20260619 02:42:14.400581 20121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f45bf4a084a4ee790312b3144923be5" candidate_uuid: "00c9712fe62f4d23a46d6270a86101e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "816a43ca3f4842079542408ac9fde61f" is_pre_election: true
I20260619 02:42:14.400739 20255 raft_consensus.cc:2454] T 0f45bf4a084a4ee790312b3144923be5 P ca02d260fb4842dcb4121f3c88a64508 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00c9712fe62f4d23a46d6270a86101e9 in term 0.
I20260619 02:42:14.400739 20121 raft_consensus.cc:2454] T 0f45bf4a084a4ee790312b3144923be5 P 816a43ca3f4842079542408ac9fde61f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00c9712fe62f4d23a46d6270a86101e9 in term 0.
I20260619 02:42:14.401001 19920 leader_election.cc:302] T 0f45bf4a084a4ee790312b3144923be5 P 00c9712fe62f4d23a46d6270a86101e9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00c9712fe62f4d23a46d6270a86101e9, ca02d260fb4842dcb4121f3c88a64508; no voters:
I20260619 02:42:14.401155 20323 raft_consensus.cc:2781] T 0f45bf4a084a4ee790312b3144923be5 P 00c9712fe62f4d23a46d6270a86101e9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:42:14.401218 20323 raft_consensus.cc:482] T 0f45bf4a084a4ee790312b3144923be5 P 00c9712fe62f4d23a46d6270a86101e9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:42:14.401253 20323 raft_consensus.cc:3037] T 0f45bf4a084a4ee790312b3144923be5 P 00c9712fe62f4d23a46d6270a86101e9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:42:14.401975 20323 raft_consensus.cc:504] T 0f45bf4a084a4ee790312b3144923be5 P 00c9712fe62f4d23a46d6270a86101e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca02d260fb4842dcb4121f3c88a64508" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 39503 } } peers { permanent_uuid: "816a43ca3f4842079542408ac9fde61f" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 46401 } } peers { permanent_uuid: "00c9712fe62f4d23a46d6270a86101e9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46275 } }
I20260619 02:42:14.402130 20323 leader_election.cc:288] T 0f45bf4a084a4ee790312b3144923be5 P 00c9712fe62f4d23a46d6270a86101e9 [CANDIDATE]: Term 1 election: Requested vote from peers ca02d260fb4842dcb4121f3c88a64508 (127.5.241.67:39503), 816a43ca3f4842079542408ac9fde61f (127.5.241.66:46401)
I20260619 02:42:14.402266 20255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f45bf4a084a4ee790312b3144923be5" candidate_uuid: "00c9712fe62f4d23a46d6270a86101e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca02d260fb4842dcb4121f3c88a64508"
I20260619 02:42:14.402287 20121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f45bf4a084a4ee790312b3144923be5" candidate_uuid: "00c9712fe62f4d23a46d6270a86101e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "816a43ca3f4842079542408ac9fde61f"
I20260619 02:42:14.402355 20121 raft_consensus.cc:3037] T 0f45bf4a084a4ee790312b3144923be5 P 816a43ca3f4842079542408ac9fde61f [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:42:14.402351 20255 raft_consensus.cc:3037] T 0f45bf4a084a4ee790312b3144923be5 P ca02d260fb4842dcb4121f3c88a64508 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:42:14.403288 20121 raft_consensus.cc:2454] T 0f45bf4a084a4ee790312b3144923be5 P 816a43ca3f4842079542408ac9fde61f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00c9712fe62f4d23a46d6270a86101e9 in term 1.
I20260619 02:42:14.403288 20255 raft_consensus.cc:2454] T 0f45bf4a084a4ee790312b3144923be5 P ca02d260fb4842dcb4121f3c88a64508 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00c9712fe62f4d23a46d6270a86101e9 in term 1.
I20260619 02:42:14.403486 19922 leader_election.cc:302] T 0f45bf4a084a4ee790312b3144923be5 P 00c9712fe62f4d23a46d6270a86101e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00c9712fe62f4d23a46d6270a86101e9, 816a43ca3f4842079542408ac9fde61f; no voters:
I20260619 02:42:14.403590 20323 raft_consensus.cc:2781] T 0f45bf4a084a4ee790312b3144923be5 P 00c9712fe62f4d23a46d6270a86101e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:42:14.403779 20323 raft_consensus.cc:686] T 0f45bf4a084a4ee790312b3144923be5 P 00c9712fe62f4d23a46d6270a86101e9 [term 1 LEADER]: Becoming Leader. State: Replica: 00c9712fe62f4d23a46d6270a86101e9, State: Running, Role: LEADER
I20260619 02:42:14.403925 20323 consensus_queue.cc:237] T 0f45bf4a084a4ee790312b3144923be5 P 00c9712fe62f4d23a46d6270a86101e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca02d260fb4842dcb4121f3c88a64508" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 39503 } } peers { permanent_uuid: "816a43ca3f4842079542408ac9fde61f" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 46401 } } peers { permanent_uuid: "00c9712fe62f4d23a46d6270a86101e9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46275 } }
I20260619 02:42:14.404855 19841 catalog_manager.cc:5495] T 0f45bf4a084a4ee790312b3144923be5 P 00c9712fe62f4d23a46d6270a86101e9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 00c9712fe62f4d23a46d6270a86101e9 (127.5.241.65). New cstate: current_term: 1 leader_uuid: "00c9712fe62f4d23a46d6270a86101e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca02d260fb4842dcb4121f3c88a64508" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 39503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "816a43ca3f4842079542408ac9fde61f" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 46401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00c9712fe62f4d23a46d6270a86101e9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46275 } health_report { overall_health: HEALTHY } } }
I20260619 02:42:14.409749 20314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:14.407537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46144 (local address 127.5.241.65:46275)
0619 02:42:14.407573 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:14.407575 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:14.407632 (+ 57us) server_negotiation.cc:407] Connection header received
0619 02:42:14.407660 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:14.407661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:14.407675 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:14.407752 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:42:14.407863 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:14.408306 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:14.409024 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:14.409238 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:14.409309 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:14.409432 (+ 123us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:42:14.409540 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:42:14.409605 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:14.409652 (+ 47us) server_negotiation.cc:299] Negotiation successful
0619 02:42:14.409671 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 02:42:14.411499 20121 raft_consensus.cc:1261] T 0f45bf4a084a4ee790312b3144923be5 P 816a43ca3f4842079542408ac9fde61f [term 1 FOLLOWER]: Refusing update from remote peer 00c9712fe62f4d23a46d6270a86101e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:42:14.411512 20255 raft_consensus.cc:1261] T 0f45bf4a084a4ee790312b3144923be5 P ca02d260fb4842dcb4121f3c88a64508 [term 1 FOLLOWER]: Refusing update from remote peer 00c9712fe62f4d23a46d6270a86101e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:42:14.411849 20323 consensus_queue.cc:1035] T 0f45bf4a084a4ee790312b3144923be5 P 00c9712fe62f4d23a46d6270a86101e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca02d260fb4842dcb4121f3c88a64508" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 39503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:42:14.411990 20325 consensus_queue.cc:1035] T 0f45bf4a084a4ee790312b3144923be5 P 00c9712fe62f4d23a46d6270a86101e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "816a43ca3f4842079542408ac9fde61f" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 46401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:42:14.414716 20330 mvcc.cc:205] Tried to move back new op lower bound from 7298404083347066880 to 7298404083319214080. Current Snapshot: MvccSnapshot[applied={T|T < 7298404083347066880}]
I20260619 02:42:14.415439 20329 mvcc.cc:205] Tried to move back new op lower bound from 7298404083347066880 to 7298404083319214080. Current Snapshot: MvccSnapshot[applied={T|T < 7298404083347066880 or (T in {7298404083347066880})}]
I20260619 02:42:14.416904 19842 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45610:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:42:14.416985 19842 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45610:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:42:14.417793 19842 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c: Sending DeleteTablet for 3 replicas of tablet 0f45bf4a084a4ee790312b3144923be5
I20260619 02:42:14.418111 20235 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f45bf4a084a4ee790312b3144923be5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:42:14 UTC) from {username='kudu', principal='kudu/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:41410
I20260619 02:42:14.418183 20101 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f45bf4a084a4ee790312b3144923be5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:42:14 UTC) from {username='kudu', principal='kudu/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:34220
I20260619 02:42:14.418329 19967 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f45bf4a084a4ee790312b3144923be5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:42:14 UTC) from {username='kudu', principal='kudu/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:46128
I20260619 02:42:14.418386 20337 tablet_replica.cc:330] T 0f45bf4a084a4ee790312b3144923be5 P ca02d260fb4842dcb4121f3c88a64508: stopping tablet replica
I20260619 02:42:14.418404 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 19904
I20260619 02:42:14.418386 20338 tablet_replica.cc:330] T 0f45bf4a084a4ee790312b3144923be5 P 816a43ca3f4842079542408ac9fde61f: stopping tablet replica
I20260619 02:42:14.418552 20338 raft_consensus.cc:2229] T 0f45bf4a084a4ee790312b3144923be5 P 816a43ca3f4842079542408ac9fde61f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:42:14.418658 20338 raft_consensus.cc:2258] T 0f45bf4a084a4ee790312b3144923be5 P 816a43ca3f4842079542408ac9fde61f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:42:14.419593 20338 ts_tablet_manager.cc:1905] T 0f45bf4a084a4ee790312b3144923be5 P 816a43ca3f4842079542408ac9fde61f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:42:14.421350 20338 ts_tablet_manager.cc:1918] T 0f45bf4a084a4ee790312b3144923be5 P 816a43ca3f4842079542408ac9fde61f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:42:14.421427 20338 log.cc:1196] T 0f45bf4a084a4ee790312b3144923be5 P 816a43ca3f4842079542408ac9fde61f: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-1/wal/wals/0f45bf4a084a4ee790312b3144923be5
I20260619 02:42:14.421701 20338 ts_tablet_manager.cc:1939] T 0f45bf4a084a4ee790312b3144923be5 P 816a43ca3f4842079542408ac9fde61f: Deleting consensus metadata
I20260619 02:42:14.422267 19828 catalog_manager.cc:4841] TS 816a43ca3f4842079542408ac9fde61f (127.5.241.66:46401): tablet 0f45bf4a084a4ee790312b3144923be5 (table test-table [id=1642464714644277a03da04d490764ae]) successfully deleted
I20260619 02:42:14.422865 20337 raft_consensus.cc:2229] T 0f45bf4a084a4ee790312b3144923be5 P ca02d260fb4842dcb4121f3c88a64508 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:42:14.422995 20337 raft_consensus.cc:2258] T 0f45bf4a084a4ee790312b3144923be5 P ca02d260fb4842dcb4121f3c88a64508 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:42:14.423378 20337 ts_tablet_manager.cc:1905] T 0f45bf4a084a4ee790312b3144923be5 P ca02d260fb4842dcb4121f3c88a64508: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 02:42:14.424656 19829 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.241.65:46275 (error 108)
I20260619 02:42:14.424930 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 20038
I20260619 02:42:14.425535 20310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:14.425046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:46275 (local address 127.0.0.1:46160)
0619 02:42:14.425465 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:14.425501 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.65:46275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402}
W20260619 02:42:14.425763 19829 catalog_manager.cc:4584] TS 00c9712fe62f4d23a46d6270a86101e9 (127.5.241.65:46275): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0f45bf4a084a4ee790312b3144923be5: Network error: Client connection negotiation failed: client connection to 127.5.241.65:46275: connect: Connection refused (error 111)
I20260619 02:42:14.430208 20337 ts_tablet_manager.cc:1918] T 0f45bf4a084a4ee790312b3144923be5 P ca02d260fb4842dcb4121f3c88a64508: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:42:14.430290 20337 log.cc:1196] T 0f45bf4a084a4ee790312b3144923be5 P ca02d260fb4842dcb4121f3c88a64508: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-2/wal/wals/0f45bf4a084a4ee790312b3144923be5
I20260619 02:42:14.430456 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 20172
I20260619 02:42:14.430655 20337 ts_tablet_manager.cc:1939] T 0f45bf4a084a4ee790312b3144923be5 P ca02d260fb4842dcb4121f3c88a64508: Deleting consensus metadata
I20260619 02:42:14.434863 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 19810
I20260619 02:42:14.440079 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:45629
--webserver_interface=127.5.241.126
--webserver_port=37257
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:42191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:45629
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:42:14.520634 20341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:42:14.520856 20341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:42:14.520879 20341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:42:14.522032 20341 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:42:14.522073 20341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:42:14.522086 20341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:42:14.522099 20341 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:42:14.522110 20341 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:42:14.523562 20341 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:42191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:45629
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:45629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.241.126
--webserver_port=37257
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20341
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:42:14.523763 20341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:42:14.523931 20341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:42:14.532586 20348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:14.532578 20350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:14.532580 20347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:42:14.532645 20341 server_base.cc:1034] running on GCE node
I20260619 02:42:14.532991 20341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:42:14.533263 20341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:42:14.534408 20341 hybrid_clock.cc:648] HybridClock initialized: now 1781836934534392 us; error 32 us; skew 500 ppm
Jun 19 02:42:14 dist-test-slave-zpbg krb5kdc[19797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836934, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:42:14.540485 20341 init.cc:377] Logged in from keytab as kudu/127.5.241.126@KRBTEST.COM (short username kudu)
I20260619 02:42:14.540980 20341 webserver.cc:460] Webserver started at http://127.5.241.126:37257/ using document root <none> and password file <none>
I20260619 02:42:14.541179 20341 fs_manager.cc:359] Metadata directory not provided
I20260619 02:42:14.541231 20341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:42:14.542644 20341 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:42:14.543408 20357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:14.543612 20341 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 02:42:14.543710 20341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "ff74bd03ec4f43e98fb7e337f818650c"
format_stamp: "Formatted at 2026-06-19 02:42:13 on dist-test-slave-zpbg"
server_key: "80fc0bb3e5cc176318063434d81fe8b9"
server_key_iv: "6386c7c0580abf292387e5fa31869cb9"
server_key_version: "encryptionkey@0"
I20260619 02:42:14.544137 20341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:42:14.569581 20341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:42:14.571995 20341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:42:14.572113 20341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:42:14.575871 20341 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.126:45629
I20260619 02:42:14.575907 20409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.126:45629 every 8 connection(s)
I20260619 02:42:14.576299 20341 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:42:14.577901 20410 sys_catalog.cc:263] Verifying existing consensus state
I20260619 02:42:14.578555 20410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c: Bootstrap starting.
I20260619 02:42:14.585232 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 20341
I20260619 02:42:14.585608 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.65:46275
--local_ip_for_outbound_sockets=127.5.241.65
--tserver_master_addrs=127.5.241.126:45629
--webserver_port=35775
--webserver_interface=127.5.241.65
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab
--principal=kudu/127.5.241.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:42191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:42:14.587378 20410 log.cc:824] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c: Log is configured to *not* fsync() on all Append() calls
I20260619 02:42:14.589404 20410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:42:14.589710 20410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c: Bootstrap complete.
I20260619 02:42:14.592058 20410 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff74bd03ec4f43e98fb7e337f818650c" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45629 } }
I20260619 02:42:14.592290 20410 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff74bd03ec4f43e98fb7e337f818650c, State: Initialized, Role: FOLLOWER
I20260619 02:42:14.592460 20410 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff74bd03ec4f43e98fb7e337f818650c" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45629 } }
I20260619 02:42:14.592520 20410 raft_consensus.cc:388] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:42:14.592586 20410 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:42:14.592633 20410 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c [term 1 FOLLOWER]: Advancing to term 2
I20260619 02:42:14.593663 20410 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff74bd03ec4f43e98fb7e337f818650c" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45629 } }
I20260619 02:42:14.593747 20410 leader_election.cc:302] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ff74bd03ec4f43e98fb7e337f818650c; no voters:
I20260619 02:42:14.593964 20410 leader_election.cc:288] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 02:42:14.594000 20414 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c [term 2 FOLLOWER]: Leader election won for term 2
I20260619 02:42:14.594179 20414 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c [term 2 LEADER]: Becoming Leader. State: Replica: ff74bd03ec4f43e98fb7e337f818650c, State: Running, Role: LEADER
I20260619 02:42:14.594244 20410 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:42:14.594324 20414 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff74bd03ec4f43e98fb7e337f818650c" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45629 } }
I20260619 02:42:14.594610 20415 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ff74bd03ec4f43e98fb7e337f818650c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff74bd03ec4f43e98fb7e337f818650c" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45629 } } }
I20260619 02:42:14.594719 20415 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c [sys.catalog]: This master's current role is: LEADER
I20260619 02:42:14.594604 20414 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff74bd03ec4f43e98fb7e337f818650c. Latest consensus state: current_term: 2 leader_uuid: "ff74bd03ec4f43e98fb7e337f818650c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff74bd03ec4f43e98fb7e337f818650c" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45629 } } }
I20260619 02:42:14.594950 20414 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c [sys.catalog]: This master's current role is: LEADER
I20260619 02:42:14.595299 20422 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:42:14.596133 20422 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:42:14.596230 20422 catalog_manager.cc:1259] Loaded cluster ID: 535ce5b6d51544439284599f7eea6080
I20260619 02:42:14.596259 20422 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:42:14.596590 20422 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260619 02:42:14.601681 20422 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260619 02:42:14.604816 20422 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:42:14.605020 20422 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c: Loaded TSK: 0
I20260619 02:42:14.605233 20422 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 02:42:14.672719 20412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:42:14.672919 20412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:42:14.672943 20412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:42:14.674125 20412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:42:14.674167 20412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:42:14.674194 20412 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.65
I20260619 02:42:14.675752 20412 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:42191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab
--principal=kudu/127.5.241.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.65:46275
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.241.65
--webserver_port=35775
--tserver_master_addrs=127.5.241.126:45629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20412
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.65
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:42:14.676004 20412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:42:14.676247 20412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:42:14.682516 20437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:14.682564 20438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:14.682653 20440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:42:14.682634 20412 server_base.cc:1034] running on GCE node
I20260619 02:42:14.682909 20412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:42:14.683158 20412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:42:14.684298 20412 hybrid_clock.cc:648] HybridClock initialized: now 1781836934684273 us; error 45 us; skew 500 ppm
Jun 19 02:42:14 dist-test-slave-zpbg krb5kdc[19797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836934, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:42:14.687037 20412 init.cc:377] Logged in from keytab as kudu/127.5.241.65@KRBTEST.COM (short username kudu)
I20260619 02:42:14.687529 20412 webserver.cc:460] Webserver started at http://127.5.241.65:35775/ using document root <none> and password file <none>
I20260619 02:42:14.687783 20412 fs_manager.cc:359] Metadata directory not provided
I20260619 02:42:14.687870 20412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:42:14.689352 20412 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:42:14.690120 20447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:14.690280 20412 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:42:14.690364 20412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-0/wal
uuid: "00c9712fe62f4d23a46d6270a86101e9"
format_stamp: "Formatted at 2026-06-19 02:42:13 on dist-test-slave-zpbg"
server_key: "f20602628e991cb38fd1cd4c336c232f"
server_key_iv: "44fddaa384111960f22326a1fa79900d"
server_key_version: "encryptionkey@0"
I20260619 02:42:14.690819 20412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:42:14.702337 20412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:42:14.704761 20412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:42:14.704926 20412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:42:14.705184 20412 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:42:14.705654 20454 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 02:42:14.707552 20412 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 02:42:14.707623 20412 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 02:42:14.707671 20412 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 02:42:14.708326 20412 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 02:42:14.708384 20412 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 02:42:14.708441 20454 tablet_bootstrap.cc:492] T 0f45bf4a084a4ee790312b3144923be5 P 00c9712fe62f4d23a46d6270a86101e9: Bootstrap starting.
I20260619 02:42:14.715735 20412 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.65:46275
I20260619 02:42:14.716111 20412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
I20260619 02:42:14.719139 20454 log.cc:824] T 0f45bf4a084a4ee790312b3144923be5 P 00c9712fe62f4d23a46d6270a86101e9: Log is configured to *not* fsync() on all Append() calls
I20260619 02:42:14.719960 20561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.65:46275 every 8 connection(s)
I20260619 02:42:14.720997 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 20412
I20260619 02:42:14.721287 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.66:46401
--local_ip_for_outbound_sockets=127.5.241.66
--tserver_master_addrs=127.5.241.126:45629
--webserver_port=33589
--webserver_interface=127.5.241.66
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab
--principal=kudu/127.5.241.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:42191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:42:14.721988 20454 tablet_bootstrap.cc:492] T 0f45bf4a084a4ee790312b3144923be5 P 00c9712fe62f4d23a46d6270a86101e9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:42:14.722285 20454 tablet_bootstrap.cc:492] T 0f45bf4a084a4ee790312b3144923be5 P 00c9712fe62f4d23a46d6270a86101e9: Bootstrap complete.
I20260619 02:42:14.723503 20454 ts_tablet_manager.cc:1397] T 0f45bf4a084a4ee790312b3144923be5 P 00c9712fe62f4d23a46d6270a86101e9: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.008s
I20260619 02:42:14.724845 20454 raft_consensus.cc:348] T 0f45bf4a084a4ee790312b3144923be5 P 00c9712fe62f4d23a46d6270a86101e9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca02d260fb4842dcb4121f3c88a64508" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 39503 } } peers { permanent_uuid: "816a43ca3f4842079542408ac9fde61f" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 46401 } } peers { permanent_uuid: "00c9712fe62f4d23a46d6270a86101e9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46275 } }
I20260619 02:42:14.725147 20454 raft_consensus.cc:729] T 0f45bf4a084a4ee790312b3144923be5 P 00c9712fe62f4d23a46d6270a86101e9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00c9712fe62f4d23a46d6270a86101e9, State: Initialized, Role: FOLLOWER
I20260619 02:42:14.725304 20454 consensus_queue.cc:260] T 0f45bf4a084a4ee790312b3144923be5 P 00c9712fe62f4d23a46d6270a86101e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca02d260fb4842dcb4121f3c88a64508" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 39503 } } peers { permanent_uuid: "816a43ca3f4842079542408ac9fde61f" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 46401 } } peers { permanent_uuid: "00c9712fe62f4d23a46d6270a86101e9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46275 } }
Jun 19 02:42:14 dist-test-slave-zpbg krb5kdc[19797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836934, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.65@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:42:14.726305 20454 ts_tablet_manager.cc:1428] T 0f45bf4a084a4ee790312b3144923be5 P 00c9712fe62f4d23a46d6270a86101e9: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260619 02:42:14.730460 20565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:14.716957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:59593 (local address 127.5.241.126:45629)
0619 02:42:14.719556 (+ 2599us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:14.719567 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:14.719618 (+ 51us) server_negotiation.cc:407] Connection header received
0619 02:42:14.719977 (+ 359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:14.719984 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:14.720140 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:14.720294 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:42:14.720968 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:14.722337 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:14.723677 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:14.724047 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:14.724167 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:14.726701 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:42:14.726711 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:42:14.726712 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:42:14.726723 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:42:14.728739 (+ 2016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:14.729152 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:14.729155 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:14.729155 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:14.729258 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:14.729511 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:14.729512 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:14.729513 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:14.729751 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:42:14.729840 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:14.730138 (+ 298us) server_negotiation.cc:299] Negotiation successful
0619 02:42:14.730269 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2548,"thread_start_us":82,"threads_started":1}
I20260619 02:42:14.730460 20564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:14.716814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.65:59593)
0619 02:42:14.718966 (+ 2152us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:14.719012 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:14.719735 (+ 723us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:14.720325 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:14.720329 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:14.720598 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:42:14.720824 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:14.720832 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:14.722492 (+ 1660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:14.722494 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:14.723527 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:14.723529 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:14.724166 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:14.724168 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:14.724313 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:14.724872 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:42:14.724881 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:42:14.726581 (+ 1700us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:42:14.728826 (+ 2245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:14.728834 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:14.728836 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:14.729064 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:14.729332 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:14.729333 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:14.729333 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:14.729442 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:14.729831 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:42:14.729837 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:42:14.729972 (+ 135us) client_negotiation.cc:769] Sending connection context
0619 02:42:14.730083 (+ 111us) client_negotiation.cc:240] Negotiation successful
0619 02:42:14.730269 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2101,"thread_start_us":118,"threads_started":1}
I20260619 02:42:14.731201 20562 heartbeater.cc:347] Connected to a master server at 127.5.241.126:45629
I20260619 02:42:14.731293 20562 heartbeater.cc:464] Registering TS with master...
I20260619 02:42:14.731458 20562 heartbeater.cc:511] Master 127.5.241.126:45629 requested a full tablet report, sending...
I20260619 02:42:14.732210 20374 ts_manager.cc:194] Registered new tserver with Master: 00c9712fe62f4d23a46d6270a86101e9 (127.5.241.65:46275)
I20260619 02:42:14.733542 20374 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.65@KRBTEST.COM'} at 127.5.241.65:59593
I20260619 02:42:14.734503 20562 heartbeater.cc:503] Master 127.5.241.126:45629 was elected leader, sending a full tablet report...
Jun 19 02:42:14 dist-test-slave-zpbg krb5kdc[19797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836934, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for kudu/127.5.241.65@KRBTEST.COM
I20260619 02:42:14.741405 20569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:14.732906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:46275 (local address 127.0.0.1:46166)
0619 02:42:14.733173 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:14.733200 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:14.733303 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:14.733487 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:14.733492 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:14.733871 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:42:14.733933 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:14.733936 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:14.735146 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:14.735147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:14.735739 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:14.735741 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:14.736247 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:14.736248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:14.736336 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:14.736839 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:42:14.736846 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:42:14.738362 (+ 1516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:42:14.740226 (+ 1864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:14.740232 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:14.740234 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:14.740431 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:14.740663 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:14.740664 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:14.740664 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:14.740724 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:14.741027 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:42:14.741033 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:42:14.741084 (+ 51us) client_negotiation.cc:769] Sending connection context
0619 02:42:14.741185 (+ 101us) client_negotiation.cc:240] Negotiation successful
0619 02:42:14.741306 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":104,"threads_started":1}
I20260619 02:42:14.741427 20570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:14.732957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46166 (local address 127.5.241.65:46275)
0619 02:42:14.733110 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:14.733114 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:14.733283 (+ 169us) server_negotiation.cc:407] Connection header received
0619 02:42:14.733379 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:14.733381 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:14.733395 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:14.733487 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:42:14.734018 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:14.735065 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:14.735854 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:14.736152 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:14.736243 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:14.738470 (+ 2227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:42:14.738478 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:42:14.738479 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:42:14.738485 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:42:14.740119 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:14.740530 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:14.740532 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:14.740533 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:14.740592 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:14.740803 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:14.740804 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:14.740804 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:14.740945 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:42:14.741033 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:14.741220 (+ 187us) server_negotiation.cc:299] Negotiation successful
0619 02:42:14.741319 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":73,"threads_started":1}
I20260619 02:42:14.741997 20489 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f45bf4a084a4ee790312b3144923be5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:42:14 UTC) from {username='kudu', principal='kudu/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:46166
I20260619 02:42:14.742532 20571 tablet_replica.cc:330] stopping tablet replica
I20260619 02:42:14.742651 20571 raft_consensus.cc:2229] T 0f45bf4a084a4ee790312b3144923be5 P 00c9712fe62f4d23a46d6270a86101e9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:42:14.742723 20571 raft_consensus.cc:2258] T 0f45bf4a084a4ee790312b3144923be5 P 00c9712fe62f4d23a46d6270a86101e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:42:14.743078 20571 ts_tablet_manager.cc:1905] T 0f45bf4a084a4ee790312b3144923be5 P 00c9712fe62f4d23a46d6270a86101e9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:42:14.745138 20571 ts_tablet_manager.cc:1918] T 0f45bf4a084a4ee790312b3144923be5 P 00c9712fe62f4d23a46d6270a86101e9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:42:14.745229 20571 log.cc:1196] T 0f45bf4a084a4ee790312b3144923be5 P 00c9712fe62f4d23a46d6270a86101e9: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-0/wal/wals/0f45bf4a084a4ee790312b3144923be5
I20260619 02:42:14.745500 20571 ts_tablet_manager.cc:1939] T 0f45bf4a084a4ee790312b3144923be5 P 00c9712fe62f4d23a46d6270a86101e9: Deleting consensus metadata
I20260619 02:42:14.746042 20361 catalog_manager.cc:4841] TS 00c9712fe62f4d23a46d6270a86101e9 (127.5.241.65:46275): tablet 0f45bf4a084a4ee790312b3144923be5 (table test-table [id=1642464714644277a03da04d490764ae]) successfully deleted
W20260619 02:42:14.810268 20567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:42:14.810518 20567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:42:14.810557 20567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:42:14.811811 20567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:42:14.811873 20567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:42:14.811920 20567 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.66
I20260619 02:42:14.813366 20567 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:42191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab
--principal=kudu/127.5.241.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.66:46401
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.241.66
--webserver_port=33589
--tserver_master_addrs=127.5.241.126:45629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.66
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:42:14.813624 20567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:42:14.813856 20567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:42:14.818872 20567 server_base.cc:1034] running on GCE node
W20260619 02:42:14.818837 20576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:14.818878 20579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:14.819072 20577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:42:14.819280 20567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:42:14.819478 20567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:42:14.820632 20567 hybrid_clock.cc:648] HybridClock initialized: now 1781836934820625 us; error 46 us; skew 500 ppm
Jun 19 02:42:14 dist-test-slave-zpbg krb5kdc[19797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836934, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:42:14.823194 20567 init.cc:377] Logged in from keytab as kudu/127.5.241.66@KRBTEST.COM (short username kudu)
I20260619 02:42:14.823654 20567 webserver.cc:460] Webserver started at http://127.5.241.66:33589/ using document root <none> and password file <none>
I20260619 02:42:14.823909 20567 fs_manager.cc:359] Metadata directory not provided
I20260619 02:42:14.823998 20567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:42:14.825392 20567 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:42:14.826027 20586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:14.826217 20567 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:42:14.826320 20567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-1/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-1/wal
uuid: "816a43ca3f4842079542408ac9fde61f"
format_stamp: "Formatted at 2026-06-19 02:42:14 on dist-test-slave-zpbg"
server_key: "7d7d61fda9659281d6367e7697c06795"
server_key_iv: "903a2433cb5cf122f717fea700121717"
server_key_version: "encryptionkey@0"
I20260619 02:42:14.826670 20567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:42:14.845646 20567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:42:14.848052 20567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:42:14.848228 20567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:42:14.848483 20567 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:42:14.848798 20567 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:42:14.848853 20567 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:14.848899 20567 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:42:14.848939 20567 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:14.854947 20567 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.66:46401
I20260619 02:42:14.855003 20699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.66:46401 every 8 connection(s)
I20260619 02:42:14.855345 20567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
Jun 19 02:42:14 dist-test-slave-zpbg krb5kdc[19797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836934, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.66@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:42:14.862244 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 20567
I20260619 02:42:14.862600 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.67:39503
--local_ip_for_outbound_sockets=127.5.241.67
--tserver_master_addrs=127.5.241.126:45629
--webserver_port=35069
--webserver_interface=127.5.241.67
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab
--principal=kudu/127.5.241.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:42191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:42:14.865357 20565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:14.856041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:46831 (local address 127.5.241.126:45629)
0619 02:42:14.856113 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:14.856115 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:14.856345 (+ 230us) server_negotiation.cc:407] Connection header received
0619 02:42:14.856885 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:14.856886 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:14.856899 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:14.856962 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:42:14.857881 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:14.858385 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:14.859443 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:14.859659 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:14.859749 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:14.861963 (+ 2214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:42:14.861965 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:42:14.861965 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:42:14.861972 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:42:14.863832 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:14.864302 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:14.864304 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:14.864305 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:14.864372 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:14.864644 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:14.864645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:14.864645 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:14.864792 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:42:14.864873 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:14.865160 (+ 287us) server_negotiation.cc:299] Negotiation successful
0619 02:42:14.865257 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 02:42:14.865572 20702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:14.856045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.66:46831)
0619 02:42:14.856225 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:14.856258 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:14.856777 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:14.857034 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:14.857037 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:14.857587 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:42:14.857779 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:14.857784 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:14.858493 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:14.858494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:14.859345 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:14.859348 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:14.859761 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:14.859762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:14.859881 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:14.860407 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:42:14.860415 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:42:14.861852 (+ 1437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:42:14.863932 (+ 2080us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:14.863939 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:14.863941 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:14.864173 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:14.864451 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:14.864452 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:14.864452 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:14.864565 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:14.864876 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:42:14.864883 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:42:14.865021 (+ 138us) client_negotiation.cc:769] Sending connection context
0619 02:42:14.865137 (+ 116us) client_negotiation.cc:240] Negotiation successful
0619 02:42:14.865318 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":92,"threads_started":1}
I20260619 02:42:14.866181 20700 heartbeater.cc:347] Connected to a master server at 127.5.241.126:45629
I20260619 02:42:14.866297 20700 heartbeater.cc:464] Registering TS with master...
I20260619 02:42:14.866494 20700 heartbeater.cc:511] Master 127.5.241.126:45629 requested a full tablet report, sending...
I20260619 02:42:14.867022 20374 ts_manager.cc:194] Registered new tserver with Master: 816a43ca3f4842079542408ac9fde61f (127.5.241.66:46401)
I20260619 02:42:14.867595 20374 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.66@KRBTEST.COM'} at 127.5.241.66:46831
W20260619 02:42:14.944967 20703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:42:14.945171 20703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:42:14.945195 20703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:42:14.946287 20703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:42:14.946322 20703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:42:14.946348 20703 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.67
I20260619 02:42:14.947933 20703 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:42191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab
--principal=kudu/127.5.241.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.67:39503
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.241.67
--webserver_port=35069
--tserver_master_addrs=127.5.241.126:45629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.67
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:42:14.948156 20703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:42:14.948343 20703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:42:14.953481 20711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:14.953455 20709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:14.953449 20708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:42:14.953590 20703 server_base.cc:1034] running on GCE node
I20260619 02:42:14.953936 20703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:42:14.954219 20703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:42:14.955389 20703 hybrid_clock.cc:648] HybridClock initialized: now 1781836934955371 us; error 42 us; skew 500 ppm
Jun 19 02:42:14 dist-test-slave-zpbg krb5kdc[19797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836934, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:42:14.957892 20703 init.cc:377] Logged in from keytab as kudu/127.5.241.67@KRBTEST.COM (short username kudu)
I20260619 02:42:14.958388 20703 webserver.cc:460] Webserver started at http://127.5.241.67:35069/ using document root <none> and password file <none>
I20260619 02:42:14.958612 20703 fs_manager.cc:359] Metadata directory not provided
I20260619 02:42:14.958698 20703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:42:14.960071 20703 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:42:14.960728 20718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:14.960930 20703 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:42:14.961033 20703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-2/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-2/wal
uuid: "ca02d260fb4842dcb4121f3c88a64508"
format_stamp: "Formatted at 2026-06-19 02:42:14 on dist-test-slave-zpbg"
server_key: "bf1afc2bede4ed86072d0561befb329f"
server_key_iv: "c0dab91368041e3d6923351a0eaf8cc1"
server_key_version: "encryptionkey@0"
I20260619 02:42:14.961383 20703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:42:14.985916 20703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:42:14.988200 20703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:42:14.988399 20703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:42:14.988654 20703 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:42:14.988973 20703 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:42:14.989030 20703 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:14.989074 20703 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:42:14.989101 20703 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:14.995440 20703 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.67:39503
I20260619 02:42:14.995478 20831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.67:39503 every 8 connection(s)
I20260619 02:42:14.995867 20703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
I20260619 02:42:14.998175 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 20703
Jun 19 02:42:15 dist-test-slave-zpbg krb5kdc[19797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836934, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.67@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:42:15.005578 20836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:14.998572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45622 (local address 127.5.241.126:45629)
0619 02:42:14.998686 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:14.998687 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:14.998736 (+ 49us) server_negotiation.cc:407] Connection header received
0619 02:42:14.998830 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:14.998831 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:14.998843 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:14.998908 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:42:14.999447 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:15.000159 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:15.000958 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:15.001187 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:15.001298 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:15.002595 (+ 1297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:42:15.002597 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:42:15.002597 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:42:15.002606 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:42:15.004309 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:15.004746 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:15.004748 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:15.004748 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:15.004812 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:15.005034 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:15.005035 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:15.005035 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:15.005137 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:42:15.005260 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:15.005360 (+ 100us) server_negotiation.cc:299] Negotiation successful
0619 02:42:15.005443 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":57,"threads_started":1}
I20260619 02:42:15.006891 20565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:14.996642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:59223 (local address 127.5.241.126:45629)
0619 02:42:14.996720 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:14.996722 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:14.996952 (+ 230us) server_negotiation.cc:407] Connection header received
0619 02:42:14.997455 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:14.997456 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:14.997467 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:14.997515 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:42:14.998705 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:14.999300 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:15.000601 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:15.000814 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:15.000905 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:15.003791 (+ 2886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:42:15.003797 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:42:15.003797 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:42:15.003804 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:42:15.005391 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:15.005869 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:15.005870 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:15.005870 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:15.005927 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:15.006256 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:15.006257 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:15.006258 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:15.006373 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:42:15.006424 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:15.006690 (+ 266us) server_negotiation.cc:299] Negotiation successful
0619 02:42:15.006756 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 02:42:15.007001 20834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:14.996617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.67:59223)
0619 02:42:14.996841 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:14.996879 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:14.997368 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:14.997576 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:14.997580 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:14.998307 (+ 727us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:42:14.998580 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:14.998588 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:14.999391 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:14.999391 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:15.000516 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:15.000519 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:15.000918 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:15.000919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:15.001041 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:15.001603 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:42:15.001614 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:42:15.003704 (+ 2090us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:42:15.005470 (+ 1766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:15.005476 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:15.005478 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:15.005775 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:15.006016 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:15.006017 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:15.006017 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:15.006155 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:15.006420 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:42:15.006425 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:42:15.006538 (+ 113us) client_negotiation.cc:769] Sending connection context
0619 02:42:15.006622 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 02:42:15.006741 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":104,"threads_started":1}
I20260619 02:42:15.007442 20832 heartbeater.cc:347] Connected to a master server at 127.5.241.126:45629
I20260619 02:42:15.007524 20832 heartbeater.cc:464] Registering TS with master...
I20260619 02:42:15.007725 20832 heartbeater.cc:511] Master 127.5.241.126:45629 requested a full tablet report, sending...
I20260619 02:42:15.008096 20374 ts_manager.cc:194] Registered new tserver with Master: ca02d260fb4842dcb4121f3c88a64508 (127.5.241.67:39503)
I20260619 02:42:15.008422 20374 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.67@KRBTEST.COM'} at 127.5.241.67:59223
I20260619 02:42:15.017431 6085 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:42:15.020335 20565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:15.017846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45626 (local address 127.5.241.126:45629)
0619 02:42:15.017935 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:15.017937 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:15.018010 (+ 73us) server_negotiation.cc:407] Connection header received
0619 02:42:15.018065 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:15.018066 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:15.018100 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:15.018167 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:42:15.018289 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:15.018822 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:15.019569 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:15.019758 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:15.019830 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:15.019963 (+ 133us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:42:15.020061 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:42:15.020125 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:15.020204 (+ 79us) server_negotiation.cc:299] Negotiation successful
0619 02:42:15.020222 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 02:42:15.020617 20374 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:45626:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:42:15.022505 20374 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:42:15.025015 20489 tablet_service.cc:1467] Processing CreateTablet for tablet 2bc943dcbdd147899aab858c990dfdd9 (DEFAULT_TABLE table=test-table [id=eac072f07ed74891ac63e3146b8fd578]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:42:15.025230 20489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bc943dcbdd147899aab858c990dfdd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:42:15.026600 20454 tablet_bootstrap.cc:492] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9: Bootstrap starting.
I20260619 02:42:15.026988 20454 tablet_bootstrap.cc:654] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9: Neither blocks nor log segments found. Creating new log.
I20260619 02:42:15.027558 20454 tablet_bootstrap.cc:492] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9: No bootstrap required, opened a new log
I20260619 02:42:15.027612 20454 ts_tablet_manager.cc:1397] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:42:15.027997 20454 raft_consensus.cc:348] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "816a43ca3f4842079542408ac9fde61f" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 46401 } } peers { permanent_uuid: "ca02d260fb4842dcb4121f3c88a64508" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 39503 } } peers { permanent_uuid: "00c9712fe62f4d23a46d6270a86101e9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46275 } }
I20260619 02:42:15.028093 20454 raft_consensus.cc:374] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:42:15.028146 20454 raft_consensus.cc:729] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00c9712fe62f4d23a46d6270a86101e9, State: Initialized, Role: FOLLOWER
I20260619 02:42:15.028195 20454 consensus_queue.cc:260] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "816a43ca3f4842079542408ac9fde61f" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 46401 } } peers { permanent_uuid: "ca02d260fb4842dcb4121f3c88a64508" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 39503 } } peers { permanent_uuid: "00c9712fe62f4d23a46d6270a86101e9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46275 } }
I20260619 02:42:15.028291 20454 ts_tablet_manager.cc:1428] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:42:15.029000 20839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:15.024726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.67:39503 (local address 127.0.0.1:41422)
0619 02:42:15.024939 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:15.024949 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:15.025000 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:15.025446 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:15.025448 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:15.025450 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:42:15.025525 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:15.025527 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:15.026499 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:15.026500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:15.027846 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:15.027849 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:15.028742 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:15.028743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:15.028809 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:15.028832 (+ 23us) client_negotiation.cc:769] Sending connection context
0619 02:42:15.028883 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 02:42:15.028923 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":159,"threads_started":1}
I20260619 02:42:15.029268 20840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:15.025049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41422 (local address 127.5.241.67:39503)
0619 02:42:15.025259 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:15.025264 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:15.025280 (+ 16us) server_negotiation.cc:407] Connection header received
0619 02:42:15.025323 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:15.025326 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:15.025343 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:15.025436 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:42:15.025608 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:15.026393 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:15.027945 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:15.028655 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:15.028725 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:15.028789 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:15.029150 (+ 361us) server_negotiation.cc:299] Negotiation successful
0619 02:42:15.029194 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":96,"threads_started":1}
I20260619 02:42:15.029960 20766 tablet_service.cc:1467] Processing CreateTablet for tablet 2bc943dcbdd147899aab858c990dfdd9 (DEFAULT_TABLE table=test-table [id=eac072f07ed74891ac63e3146b8fd578]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:42:15.030369 20766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bc943dcbdd147899aab858c990dfdd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:42:15.030547 20569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:15.024644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.66:46401 (local address 127.0.0.1:34228)
0619 02:42:15.024693 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:15.024703 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:15.024740 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:15.026849 (+ 2109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:15.026850 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:15.026852 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:42:15.026897 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:15.026899 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:15.028138 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:15.028139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:15.029404 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:15.029407 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:15.030313 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:15.030313 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:15.030366 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:15.030369 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:42:15.030411 (+ 42us) client_negotiation.cc:240] Negotiation successful
0619 02:42:15.030478 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260619 02:42:15.030596 20838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:15.024739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34228 (local address 127.5.241.66:46401)
0619 02:42:15.026663 (+ 1924us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:15.026667 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:15.026678 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:42:15.026718 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:15.026720 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:15.026736 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:15.026852 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:42:15.026963 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:15.028043 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:15.029475 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:15.030239 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:15.030313 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:15.030372 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:15.030477 (+ 105us) server_negotiation.cc:299] Negotiation successful
0619 02:42:15.030522 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1884,"thread_start_us":99,"threads_started":1}
I20260619 02:42:15.033131 20842 tablet_bootstrap.cc:492] T 2bc943dcbdd147899aab858c990dfdd9 P ca02d260fb4842dcb4121f3c88a64508: Bootstrap starting.
I20260619 02:42:15.033756 20842 tablet_bootstrap.cc:654] T 2bc943dcbdd147899aab858c990dfdd9 P ca02d260fb4842dcb4121f3c88a64508: Neither blocks nor log segments found. Creating new log.
I20260619 02:42:15.033855 20634 tablet_service.cc:1467] Processing CreateTablet for tablet 2bc943dcbdd147899aab858c990dfdd9 (DEFAULT_TABLE table=test-table [id=eac072f07ed74891ac63e3146b8fd578]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:42:15.034192 20634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bc943dcbdd147899aab858c990dfdd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:42:15.034307 20842 log.cc:824] T 2bc943dcbdd147899aab858c990dfdd9 P ca02d260fb4842dcb4121f3c88a64508: Log is configured to *not* fsync() on all Append() calls
I20260619 02:42:15.035033 20842 tablet_bootstrap.cc:492] T 2bc943dcbdd147899aab858c990dfdd9 P ca02d260fb4842dcb4121f3c88a64508: No bootstrap required, opened a new log
I20260619 02:42:15.035089 20842 ts_tablet_manager.cc:1397] T 2bc943dcbdd147899aab858c990dfdd9 P ca02d260fb4842dcb4121f3c88a64508: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:42:15.036446 20842 raft_consensus.cc:348] T 2bc943dcbdd147899aab858c990dfdd9 P ca02d260fb4842dcb4121f3c88a64508 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "816a43ca3f4842079542408ac9fde61f" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 46401 } } peers { permanent_uuid: "ca02d260fb4842dcb4121f3c88a64508" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 39503 } } peers { permanent_uuid: "00c9712fe62f4d23a46d6270a86101e9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46275 } }
I20260619 02:42:15.036641 20842 raft_consensus.cc:374] T 2bc943dcbdd147899aab858c990dfdd9 P ca02d260fb4842dcb4121f3c88a64508 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:42:15.036679 20842 raft_consensus.cc:729] T 2bc943dcbdd147899aab858c990dfdd9 P ca02d260fb4842dcb4121f3c88a64508 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca02d260fb4842dcb4121f3c88a64508, State: Initialized, Role: FOLLOWER
I20260619 02:42:15.036805 20842 consensus_queue.cc:260] T 2bc943dcbdd147899aab858c990dfdd9 P ca02d260fb4842dcb4121f3c88a64508 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "816a43ca3f4842079542408ac9fde61f" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 46401 } } peers { permanent_uuid: "ca02d260fb4842dcb4121f3c88a64508" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 39503 } } peers { permanent_uuid: "00c9712fe62f4d23a46d6270a86101e9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46275 } }
I20260619 02:42:15.037106 20842 ts_tablet_manager.cc:1428] T 2bc943dcbdd147899aab858c990dfdd9 P ca02d260fb4842dcb4121f3c88a64508: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 02:42:15.037119 20832 heartbeater.cc:503] Master 127.5.241.126:45629 was elected leader, sending a full tablet report...
I20260619 02:42:15.037323 20845 tablet_bootstrap.cc:492] T 2bc943dcbdd147899aab858c990dfdd9 P 816a43ca3f4842079542408ac9fde61f: Bootstrap starting.
I20260619 02:42:15.037974 20845 tablet_bootstrap.cc:654] T 2bc943dcbdd147899aab858c990dfdd9 P 816a43ca3f4842079542408ac9fde61f: Neither blocks nor log segments found. Creating new log.
I20260619 02:42:15.038489 20845 log.cc:824] T 2bc943dcbdd147899aab858c990dfdd9 P 816a43ca3f4842079542408ac9fde61f: Log is configured to *not* fsync() on all Append() calls
I20260619 02:42:15.039273 20845 tablet_bootstrap.cc:492] T 2bc943dcbdd147899aab858c990dfdd9 P 816a43ca3f4842079542408ac9fde61f: No bootstrap required, opened a new log
I20260619 02:42:15.039330 20845 ts_tablet_manager.cc:1397] T 2bc943dcbdd147899aab858c990dfdd9 P 816a43ca3f4842079542408ac9fde61f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:42:15.040665 20845 raft_consensus.cc:348] T 2bc943dcbdd147899aab858c990dfdd9 P 816a43ca3f4842079542408ac9fde61f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "816a43ca3f4842079542408ac9fde61f" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 46401 } } peers { permanent_uuid: "ca02d260fb4842dcb4121f3c88a64508" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 39503 } } peers { permanent_uuid: "00c9712fe62f4d23a46d6270a86101e9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46275 } }
I20260619 02:42:15.040874 20845 raft_consensus.cc:374] T 2bc943dcbdd147899aab858c990dfdd9 P 816a43ca3f4842079542408ac9fde61f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:42:15.040925 20845 raft_consensus.cc:729] T 2bc943dcbdd147899aab858c990dfdd9 P 816a43ca3f4842079542408ac9fde61f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 816a43ca3f4842079542408ac9fde61f, State: Initialized, Role: FOLLOWER
I20260619 02:42:15.041050 20845 consensus_queue.cc:260] T 2bc943dcbdd147899aab858c990dfdd9 P 816a43ca3f4842079542408ac9fde61f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "816a43ca3f4842079542408ac9fde61f" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 46401 } } peers { permanent_uuid: "ca02d260fb4842dcb4121f3c88a64508" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 39503 } } peers { permanent_uuid: "00c9712fe62f4d23a46d6270a86101e9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46275 } }
I20260619 02:42:15.041324 20845 ts_tablet_manager.cc:1428] T 2bc943dcbdd147899aab858c990dfdd9 P 816a43ca3f4842079542408ac9fde61f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:42:15.041368 20700 heartbeater.cc:503] Master 127.5.241.126:45629 was elected leader, sending a full tablet report...
I20260619 02:42:15.077767 20568 raft_consensus.cc:482] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:42:15.077965 20568 raft_consensus.cc:504] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "816a43ca3f4842079542408ac9fde61f" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 46401 } } peers { permanent_uuid: "ca02d260fb4842dcb4121f3c88a64508" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 39503 } } peers { permanent_uuid: "00c9712fe62f4d23a46d6270a86101e9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46275 } }
I20260619 02:42:15.078358 20568 leader_election.cc:288] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 816a43ca3f4842079542408ac9fde61f (127.5.241.66:46401), ca02d260fb4842dcb4121f3c88a64508 (127.5.241.67:39503)
I20260619 02:42:15.081328 20564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:15.078564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.67:39503 (local address 127.5.241.65:44089)
0619 02:42:15.078625 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:15.078638 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:15.078699 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:15.078943 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:15.078944 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:15.078946 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:42:15.079027 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:15.079029 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:15.079684 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:15.079685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:15.080479 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:15.080481 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:15.081052 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:15.081054 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:15.081138 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:15.081165 (+ 27us) client_negotiation.cc:769] Sending connection context
0619 02:42:15.081224 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 02:42:15.081274 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260619 02:42:15.081408 20840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:15.078703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:44089 (local address 127.5.241.67:39503)
0619 02:42:15.078781 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:15.078783 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:15.078794 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:42:15.078825 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:15.078826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:15.078838 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:15.078903 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:42:15.079100 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:15.079608 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:15.080566 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:15.080974 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:15.081030 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:15.081037 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:15.081251 (+ 214us) server_negotiation.cc:299] Negotiation successful
0619 02:42:15.081300 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 02:42:15.081533 20838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:15.078709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:58163 (local address 127.5.241.66:46401)
0619 02:42:15.078785 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:15.078787 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:15.078920 (+ 133us) server_negotiation.cc:407] Connection header received
0619 02:42:15.078951 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:15.078952 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:15.078962 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:15.079037 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:42:15.079207 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:15.079703 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:15.080656 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:15.081060 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:15.081132 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:15.081140 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:15.081399 (+ 259us) server_negotiation.cc:299] Negotiation successful
0619 02:42:15.081443 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 02:42:15.081553 20848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:15.078579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.66:46401 (local address 127.5.241.65:58163)
0619 02:42:15.078820 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:15.078830 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:15.078894 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:15.079045 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:15.079046 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:15.079047 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:42:15.079135 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:15.079136 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:15.079826 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:15.079827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:15.080573 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:15.080575 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:15.081180 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:15.081181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:15.081303 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:15.081308 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 02:42:15.081385 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 02:42:15.081442 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"mutex_wait_us":31,"thread_start_us":120,"threads_started":1}
I20260619 02:42:15.081776 20786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bc943dcbdd147899aab858c990dfdd9" candidate_uuid: "00c9712fe62f4d23a46d6270a86101e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca02d260fb4842dcb4121f3c88a64508" is_pre_election: true
I20260619 02:42:15.081977 20786 raft_consensus.cc:2454] T 2bc943dcbdd147899aab858c990dfdd9 P ca02d260fb4842dcb4121f3c88a64508 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00c9712fe62f4d23a46d6270a86101e9 in term 0.
I20260619 02:42:15.081938 20654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bc943dcbdd147899aab858c990dfdd9" candidate_uuid: "00c9712fe62f4d23a46d6270a86101e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "816a43ca3f4842079542408ac9fde61f" is_pre_election: true
I20260619 02:42:15.082086 20654 raft_consensus.cc:2454] T 2bc943dcbdd147899aab858c990dfdd9 P 816a43ca3f4842079542408ac9fde61f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00c9712fe62f4d23a46d6270a86101e9 in term 0.
I20260619 02:42:15.082229 20448 leader_election.cc:302] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00c9712fe62f4d23a46d6270a86101e9, ca02d260fb4842dcb4121f3c88a64508; no voters:
I20260619 02:42:15.082405 20568 raft_consensus.cc:2781] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:42:15.082477 20568 raft_consensus.cc:482] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:42:15.082546 20568 raft_consensus.cc:3037] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:42:15.083477 20568 raft_consensus.cc:504] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "816a43ca3f4842079542408ac9fde61f" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 46401 } } peers { permanent_uuid: "ca02d260fb4842dcb4121f3c88a64508" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 39503 } } peers { permanent_uuid: "00c9712fe62f4d23a46d6270a86101e9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46275 } }
I20260619 02:42:15.083654 20568 leader_election.cc:288] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9 [CANDIDATE]: Term 1 election: Requested vote from peers 816a43ca3f4842079542408ac9fde61f (127.5.241.66:46401), ca02d260fb4842dcb4121f3c88a64508 (127.5.241.67:39503)
I20260619 02:42:15.083817 20654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bc943dcbdd147899aab858c990dfdd9" candidate_uuid: "00c9712fe62f4d23a46d6270a86101e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "816a43ca3f4842079542408ac9fde61f"
I20260619 02:42:15.083890 20654 raft_consensus.cc:3037] T 2bc943dcbdd147899aab858c990dfdd9 P 816a43ca3f4842079542408ac9fde61f [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:42:15.084574 20654 raft_consensus.cc:2454] T 2bc943dcbdd147899aab858c990dfdd9 P 816a43ca3f4842079542408ac9fde61f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00c9712fe62f4d23a46d6270a86101e9 in term 1.
I20260619 02:42:15.084709 20786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bc943dcbdd147899aab858c990dfdd9" candidate_uuid: "00c9712fe62f4d23a46d6270a86101e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca02d260fb4842dcb4121f3c88a64508"
I20260619 02:42:15.084767 20450 leader_election.cc:302] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00c9712fe62f4d23a46d6270a86101e9, 816a43ca3f4842079542408ac9fde61f; no voters:
I20260619 02:42:15.084800 20786 raft_consensus.cc:3037] T 2bc943dcbdd147899aab858c990dfdd9 P ca02d260fb4842dcb4121f3c88a64508 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:42:15.084851 20568 raft_consensus.cc:2781] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:42:15.085012 20568 raft_consensus.cc:686] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9 [term 1 LEADER]: Becoming Leader. State: Replica: 00c9712fe62f4d23a46d6270a86101e9, State: Running, Role: LEADER
I20260619 02:42:15.085093 20568 consensus_queue.cc:237] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "816a43ca3f4842079542408ac9fde61f" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 46401 } } peers { permanent_uuid: "ca02d260fb4842dcb4121f3c88a64508" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 39503 } } peers { permanent_uuid: "00c9712fe62f4d23a46d6270a86101e9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46275 } }
I20260619 02:42:15.085773 20786 raft_consensus.cc:2454] T 2bc943dcbdd147899aab858c990dfdd9 P ca02d260fb4842dcb4121f3c88a64508 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00c9712fe62f4d23a46d6270a86101e9 in term 1.
I20260619 02:42:15.085937 20374 catalog_manager.cc:5495] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 00c9712fe62f4d23a46d6270a86101e9 (127.5.241.65). New cstate: current_term: 1 leader_uuid: "00c9712fe62f4d23a46d6270a86101e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "816a43ca3f4842079542408ac9fde61f" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 46401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca02d260fb4842dcb4121f3c88a64508" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 39503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00c9712fe62f4d23a46d6270a86101e9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46275 } health_report { overall_health: HEALTHY } } }
I20260619 02:42:15.102371 20570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:15.099873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46176 (local address 127.5.241.65:46275)
0619 02:42:15.099929 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:15.099932 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:15.099943 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:42:15.099989 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:15.099990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:15.100007 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:15.100086 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:42:15.100195 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:15.100748 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:15.101520 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:15.101765 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:15.101832 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:15.102021 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:42:15.102127 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:42:15.102222 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:15.102262 (+ 40us) server_negotiation.cc:299] Negotiation successful
0619 02:42:15.102284 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 02:42:15.104065 20654 raft_consensus.cc:1261] T 2bc943dcbdd147899aab858c990dfdd9 P 816a43ca3f4842079542408ac9fde61f [term 1 FOLLOWER]: Refusing update from remote peer 00c9712fe62f4d23a46d6270a86101e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:42:15.104065 20786 raft_consensus.cc:1261] T 2bc943dcbdd147899aab858c990dfdd9 P ca02d260fb4842dcb4121f3c88a64508 [term 1 FOLLOWER]: Refusing update from remote peer 00c9712fe62f4d23a46d6270a86101e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:42:15.104439 20568 consensus_queue.cc:1035] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "816a43ca3f4842079542408ac9fde61f" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 46401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:42:15.104643 20849 consensus_queue.cc:1035] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca02d260fb4842dcb4121f3c88a64508" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 39503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 02:42:15.105906 20701 tablet.cc:2369] T 2bc943dcbdd147899aab858c990dfdd9 P 816a43ca3f4842079542408ac9fde61f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:42:15.107329 20851 mvcc.cc:205] Tried to move back new op lower bound from 7298404086183321600 to 7298404086109007872. Current Snapshot: MvccSnapshot[applied={T|T < 7298404086183321600 or (T in {7298404086183321600})}]
I20260619 02:42:15.108949 20374 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:45626:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:42:15.109042 20374 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:45626:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:42:15.109727 20374 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c: Sending DeleteTablet for 3 replicas of tablet 2bc943dcbdd147899aab858c990dfdd9
I20260619 02:42:15.110049 20634 tablet_service.cc:1514] Processing DeleteTablet for tablet 2bc943dcbdd147899aab858c990dfdd9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:42:15 UTC) from {username='kudu', principal='kudu/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:34228
I20260619 02:42:15.110049 20766 tablet_service.cc:1514] Processing DeleteTablet for tablet 2bc943dcbdd147899aab858c990dfdd9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:42:15 UTC) from {username='kudu', principal='kudu/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:41422
I20260619 02:42:15.110242 20489 tablet_service.cc:1514] Processing DeleteTablet for tablet 2bc943dcbdd147899aab858c990dfdd9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:42:15 UTC) from {username='kudu', principal='kudu/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:46166
I20260619 02:42:15.110284 20861 tablet_replica.cc:330] T 2bc943dcbdd147899aab858c990dfdd9 P 816a43ca3f4842079542408ac9fde61f: stopping tablet replica
I20260619 02:42:15.110348 20571 tablet_replica.cc:330] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9: stopping tablet replica
I20260619 02:42:15.110352 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 20412
I20260619 02:42:15.110405 20571 raft_consensus.cc:2229] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:42:15.110469 20861 raft_consensus.cc:2229] T 2bc943dcbdd147899aab858c990dfdd9 P 816a43ca3f4842079542408ac9fde61f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:42:15.110489 20571 raft_consensus.cc:2258] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:42:15.110513 20862 tablet_replica.cc:330] T 2bc943dcbdd147899aab858c990dfdd9 P ca02d260fb4842dcb4121f3c88a64508: stopping tablet replica
I20260619 02:42:15.110631 20861 raft_consensus.cc:2258] T 2bc943dcbdd147899aab858c990dfdd9 P 816a43ca3f4842079542408ac9fde61f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:42:15.110649 20862 raft_consensus.cc:2229] T 2bc943dcbdd147899aab858c990dfdd9 P ca02d260fb4842dcb4121f3c88a64508 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:42:15.110788 20862 raft_consensus.cc:2258] T 2bc943dcbdd147899aab858c990dfdd9 P ca02d260fb4842dcb4121f3c88a64508 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:42:15.111265 20861 ts_tablet_manager.cc:1905] T 2bc943dcbdd147899aab858c990dfdd9 P 816a43ca3f4842079542408ac9fde61f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:42:15.111835 20862 ts_tablet_manager.cc:1905] T 2bc943dcbdd147899aab858c990dfdd9 P ca02d260fb4842dcb4121f3c88a64508: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:42:15.113188 20861 ts_tablet_manager.cc:1918] T 2bc943dcbdd147899aab858c990dfdd9 P 816a43ca3f4842079542408ac9fde61f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:42:15.113284 20861 log.cc:1196] T 2bc943dcbdd147899aab858c990dfdd9 P 816a43ca3f4842079542408ac9fde61f: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-1/wal/wals/2bc943dcbdd147899aab858c990dfdd9
I20260619 02:42:15.113531 20861 ts_tablet_manager.cc:1939] T 2bc943dcbdd147899aab858c990dfdd9 P 816a43ca3f4842079542408ac9fde61f: Deleting consensus metadata
I20260619 02:42:15.113719 20862 ts_tablet_manager.cc:1918] T 2bc943dcbdd147899aab858c990dfdd9 P ca02d260fb4842dcb4121f3c88a64508: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:42:15.113791 20862 log.cc:1196] T 2bc943dcbdd147899aab858c990dfdd9 P ca02d260fb4842dcb4121f3c88a64508: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-2/wal/wals/2bc943dcbdd147899aab858c990dfdd9
I20260619 02:42:15.113936 20360 catalog_manager.cc:4841] TS 816a43ca3f4842079542408ac9fde61f (127.5.241.66:46401): tablet 2bc943dcbdd147899aab858c990dfdd9 (table test-table [id=eac072f07ed74891ac63e3146b8fd578]) successfully deleted
I20260619 02:42:15.114058 20862 ts_tablet_manager.cc:1939] T 2bc943dcbdd147899aab858c990dfdd9 P ca02d260fb4842dcb4121f3c88a64508: Deleting consensus metadata
I20260619 02:42:15.114502 20358 catalog_manager.cc:4841] TS ca02d260fb4842dcb4121f3c88a64508 (127.5.241.67:39503): tablet 2bc943dcbdd147899aab858c990dfdd9 (table test-table [id=eac072f07ed74891ac63e3146b8fd578]) successfully deleted
W20260619 02:42:15.116349 20361 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.241.65:46275 (error 108)
I20260619 02:42:15.116644 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 20567
I20260619 02:42:15.117293 20569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:15.116713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:46275 (local address 127.0.0.1:46184)
0619 02:42:15.117208 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:15.117254 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.65:46275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":471}
W20260619 02:42:15.117401 20361 catalog_manager.cc:4584] TS 00c9712fe62f4d23a46d6270a86101e9 (127.5.241.65:46275): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2bc943dcbdd147899aab858c990dfdd9: Network error: Client connection negotiation failed: client connection to 127.5.241.65:46275: connect: Connection refused (error 111)
I20260619 02:42:15.122515 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 20703
I20260619 02:42:15.126988 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 20341
I20260619 02:42:15.132463 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:45629
--webserver_interface=127.5.241.126
--webserver_port=37257
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:42191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:45629
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:42:15.213140 20864 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:42:15.213375 20864 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:42:15.213413 20864 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:42:15.214586 20864 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:42:15.214653 20864 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:42:15.214679 20864 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:42:15.214706 20864 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:42:15.214740 20864 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:42:15.216231 20864 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:42191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:45629
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:45629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.241.126
--webserver_port=37257
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20864
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:42:15.216501 20864 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:42:15.216768 20864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:42:15.221860 20872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:15.221841 20870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:15.221828 20869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:42:15.222044 20864 server_base.cc:1034] running on GCE node
I20260619 02:42:15.222332 20864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:42:15.222641 20864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:42:15.223862 20864 hybrid_clock.cc:648] HybridClock initialized: now 1781836935223853 us; error 34 us; skew 500 ppm
Jun 19 02:42:15 dist-test-slave-zpbg krb5kdc[19797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836935, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:42:15.226441 20864 init.cc:377] Logged in from keytab as kudu/127.5.241.126@KRBTEST.COM (short username kudu)
I20260619 02:42:15.226930 20864 webserver.cc:460] Webserver started at http://127.5.241.126:37257/ using document root <none> and password file <none>
I20260619 02:42:15.227164 20864 fs_manager.cc:359] Metadata directory not provided
I20260619 02:42:15.227231 20864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:42:15.228834 20864 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:42:15.229641 20879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:15.229914 20864 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:42:15.230022 20864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "ff74bd03ec4f43e98fb7e337f818650c"
format_stamp: "Formatted at 2026-06-19 02:42:13 on dist-test-slave-zpbg"
server_key: "80fc0bb3e5cc176318063434d81fe8b9"
server_key_iv: "6386c7c0580abf292387e5fa31869cb9"
server_key_version: "encryptionkey@0"
I20260619 02:42:15.230381 20864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:42:15.248224 20864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:42:15.250298 20864 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:42:15.250469 20864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:42:15.254531 20864 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.126:45629
I20260619 02:42:15.254565 20931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.126:45629 every 8 connection(s)
I20260619 02:42:15.254997 20864 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:42:15.256589 20932 sys_catalog.cc:263] Verifying existing consensus state
I20260619 02:42:15.257171 20932 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c: Bootstrap starting.
I20260619 02:42:15.257423 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 20864
I20260619 02:42:15.257725 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.65:46275
--local_ip_for_outbound_sockets=127.5.241.65
--tserver_master_addrs=127.5.241.126:45629
--webserver_port=35775
--webserver_interface=127.5.241.65
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab
--principal=kudu/127.5.241.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:42191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:42:15.266153 20932 log.cc:824] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c: Log is configured to *not* fsync() on all Append() calls
I20260619 02:42:15.269212 20932 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:42:15.269558 20932 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c: Bootstrap complete.
I20260619 02:42:15.272779 20932 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff74bd03ec4f43e98fb7e337f818650c" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45629 } }
I20260619 02:42:15.273090 20932 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff74bd03ec4f43e98fb7e337f818650c, State: Initialized, Role: FOLLOWER
I20260619 02:42:15.273245 20932 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff74bd03ec4f43e98fb7e337f818650c" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45629 } }
I20260619 02:42:15.273303 20932 raft_consensus.cc:388] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:42:15.273353 20932 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:42:15.273401 20932 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c [term 2 FOLLOWER]: Advancing to term 3
I20260619 02:42:15.274714 20932 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff74bd03ec4f43e98fb7e337f818650c" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45629 } }
I20260619 02:42:15.274842 20932 leader_election.cc:302] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ff74bd03ec4f43e98fb7e337f818650c; no voters:
I20260619 02:42:15.275041 20932 leader_election.cc:288] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c [CANDIDATE]: Term 3 election: Requested vote from peers
I20260619 02:42:15.275156 20936 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c [term 3 FOLLOWER]: Leader election won for term 3
I20260619 02:42:15.275312 20936 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c [term 3 LEADER]: Becoming Leader. State: Replica: ff74bd03ec4f43e98fb7e337f818650c, State: Running, Role: LEADER
I20260619 02:42:15.275331 20932 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:42:15.275409 20936 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff74bd03ec4f43e98fb7e337f818650c" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45629 } }
I20260619 02:42:15.275652 20937 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "ff74bd03ec4f43e98fb7e337f818650c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff74bd03ec4f43e98fb7e337f818650c" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45629 } } }
I20260619 02:42:15.275768 20937 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c [sys.catalog]: This master's current role is: LEADER
I20260619 02:42:15.275986 20938 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff74bd03ec4f43e98fb7e337f818650c. Latest consensus state: current_term: 3 leader_uuid: "ff74bd03ec4f43e98fb7e337f818650c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff74bd03ec4f43e98fb7e337f818650c" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45629 } } }
I20260619 02:42:15.276116 20942 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:42:15.276116 20938 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff74bd03ec4f43e98fb7e337f818650c [sys.catalog]: This master's current role is: LEADER
I20260619 02:42:15.277033 20942 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:42:15.277134 20942 catalog_manager.cc:1259] Loaded cluster ID: 535ce5b6d51544439284599f7eea6080
I20260619 02:42:15.277158 20942 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260619 02:42:15.277546 20942 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fa55ab079ad google::LogMessage::Fail() at ??:0
@ 0x7fa55ab0a88a google::LogMessage::SendToLog() at ??:0
@ 0x7fa55ab074de google::LogMessage::Flush() at ??:0
@ 0x7fa55ab08a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fa562a2d0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fa562a368b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fa55c9d50d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fa55c9ca954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fa55d82e6db start_thread at ??:0
@ 0x7fa55960561f clone at ??:0
W20260619 02:42:15.356012 20934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:42:15.356215 20934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:42:15.356240 20934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:42:15.357450 20934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:42:15.357494 20934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:42:15.357522 20934 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.65
I20260619 02:42:15.359016 20934 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:42191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab
--principal=kudu/127.5.241.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.65:46275
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.241.65
--webserver_port=35775
--tserver_master_addrs=127.5.241.126:45629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20934
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.65
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:42:15.359257 20934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:42:15.359508 20934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:42:15.364466 20969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:15.364598 20968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:15.364662 20971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:42:15.364964 20934 server_base.cc:1034] running on GCE node
I20260619 02:42:15.365238 20934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:42:15.365537 20934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:42:15.366715 20934 hybrid_clock.cc:648] HybridClock initialized: now 1781836935366698 us; error 35 us; skew 500 ppm
Jun 19 02:42:15 dist-test-slave-zpbg krb5kdc[19797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836935, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:42:15.370079 20934 init.cc:377] Logged in from keytab as kudu/127.5.241.65@KRBTEST.COM (short username kudu)
I20260619 02:42:15.370656 20934 webserver.cc:460] Webserver started at http://127.5.241.65:35775/ using document root <none> and password file <none>
I20260619 02:42:15.370951 20934 fs_manager.cc:359] Metadata directory not provided
I20260619 02:42:15.371042 20934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:42:15.372852 20934 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:42:15.373687 20978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:15.373919 20934 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:42:15.374037 20934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-0/wal
uuid: "00c9712fe62f4d23a46d6270a86101e9"
format_stamp: "Formatted at 2026-06-19 02:42:13 on dist-test-slave-zpbg"
server_key: "f20602628e991cb38fd1cd4c336c232f"
server_key_iv: "44fddaa384111960f22326a1fa79900d"
server_key_version: "encryptionkey@0"
I20260619 02:42:15.374502 20934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:42:15.398453 20934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:42:15.401216 20934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:42:15.401425 20934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:42:15.401799 20934 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:42:15.402315 20985 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 02:42:15.404052 20934 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 02:42:15.404155 20934 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 02:42:15.404210 20934 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 02:42:15.404986 20934 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 02:42:15.405035 20934 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 02:42:15.405118 20985 tablet_bootstrap.cc:492] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9: Bootstrap starting.
I20260619 02:42:15.413249 20934 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.65:46275
I20260619 02:42:15.413463 21092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.65:46275 every 8 connection(s)
I20260619 02:42:15.413765 20934 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
I20260619 02:42:15.415562 20985 log.cc:824] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9: Log is configured to *not* fsync() on all Append() calls
I20260619 02:42:15.417541 20985 tablet_bootstrap.cc:492] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:42:15.417829 20985 tablet_bootstrap.cc:492] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9: Bootstrap complete.
I20260619 02:42:15.418740 20985 ts_tablet_manager.cc:1397] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.003s
I20260619 02:42:15.419791 20985 raft_consensus.cc:348] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "816a43ca3f4842079542408ac9fde61f" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 46401 } } peers { permanent_uuid: "ca02d260fb4842dcb4121f3c88a64508" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 39503 } } peers { permanent_uuid: "00c9712fe62f4d23a46d6270a86101e9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46275 } }
I20260619 02:42:15.420017 20985 raft_consensus.cc:729] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00c9712fe62f4d23a46d6270a86101e9, State: Initialized, Role: FOLLOWER
I20260619 02:42:15.420147 20985 consensus_queue.cc:260] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "816a43ca3f4842079542408ac9fde61f" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 46401 } } peers { permanent_uuid: "ca02d260fb4842dcb4121f3c88a64508" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 39503 } } peers { permanent_uuid: "00c9712fe62f4d23a46d6270a86101e9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46275 } }
I20260619 02:42:15.420415 20985 ts_tablet_manager.cc:1428] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:42:15.423831 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 20934
I20260619 02:42:15.424114 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.66:46401
--local_ip_for_outbound_sockets=127.5.241.66
--tserver_master_addrs=127.5.241.126:45629
--webserver_port=33589
--webserver_interface=127.5.241.66
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab
--principal=kudu/127.5.241.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:42191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:42:15.504693 21098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:42:15.504917 21098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:42:15.504941 21098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:42:15.506134 21098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:42:15.506174 21098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:42:15.506201 21098 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.66
I20260619 02:42:15.507686 21098 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:42191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab
--principal=kudu/127.5.241.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.66:46401
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.241.66
--webserver_port=33589
--tserver_master_addrs=127.5.241.126:45629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21098
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.66
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:42:15.507954 21098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:42:15.508244 21098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:42:15.512741 21095 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:42:15.414558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.65:36107)
0619 02:42:15.414814 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:15.414868 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:15.415636 (+ 768us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:15.512511 (+ 96875us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45629: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":111,"threads_started":1}
W20260619 02:42:15.512948 21093 heartbeater.cc:644] Failed to heartbeat to 127.5.241.126:45629 (0 consecutive failures): Network error: Failed to ping master at 127.5.241.126:45629: Client connection negotiation failed: client connection to 127.5.241.126:45629: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 02:42:15.513279 21095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:15.513154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.65:56805)
0619 02:42:15.513212 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:15.513260 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260619 02:42:15.513641 21095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:15.513562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.65:43279)
0619 02:42:15.513604 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:15.513627 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260619 02:42:15.514122 21093 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 02:42:15.515201 21104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:15.515208 21106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:15.515213 21103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:42:15.515339 21098 server_base.cc:1034] running on GCE node
I20260619 02:42:15.515594 21098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:42:15.515837 21098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:42:15.516978 21098 hybrid_clock.cc:648] HybridClock initialized: now 1781836935516953 us; error 37 us; skew 500 ppm
Jun 19 02:42:15 dist-test-slave-zpbg krb5kdc[19797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836935, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:42:15.519415 21098 init.cc:377] Logged in from keytab as kudu/127.5.241.66@KRBTEST.COM (short username kudu)
I20260619 02:42:15.519842 21098 webserver.cc:460] Webserver started at http://127.5.241.66:33589/ using document root <none> and password file <none>
I20260619 02:42:15.520076 21098 fs_manager.cc:359] Metadata directory not provided
I20260619 02:42:15.520121 21098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:42:15.521404 21098 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:42:15.521960 21113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:15.522172 21098 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 02:42:15.522253 21098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-1/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-1/wal
uuid: "816a43ca3f4842079542408ac9fde61f"
format_stamp: "Formatted at 2026-06-19 02:42:14 on dist-test-slave-zpbg"
server_key: "7d7d61fda9659281d6367e7697c06795"
server_key_iv: "903a2433cb5cf122f717fea700121717"
server_key_version: "encryptionkey@0"
I20260619 02:42:15.522591 21098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:42:15.564857 21098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:42:15.566905 21098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:42:15.567022 21098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:42:15.567214 21098 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:42:15.567552 21098 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:42:15.567584 21098 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:15.567643 21098 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:42:15.567672 21098 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:15.573478 21098 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.66:46401
I20260619 02:42:15.573571 21226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.66:46401 every 8 connection(s)
I20260619 02:42:15.573901 21098 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
I20260619 02:42:15.574962 21229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:15.574561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.66:57499)
0619 02:42:15.574705 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:15.574817 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":65,"threads_started":1}
W20260619 02:42:15.575363 21227 heartbeater.cc:644] Failed to heartbeat to 127.5.241.126:45629 (0 consecutive failures): Network error: Failed to ping master at 127.5.241.126:45629: Client connection negotiation failed: client connection to 127.5.241.126:45629: connect: Connection refused (error 111)
I20260619 02:42:15.575618 21229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:15.575533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.66:47875)
0619 02:42:15.575577 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:15.575602 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260619 02:42:15.575852 21229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:15.575805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.66:58247)
0619 02:42:15.575825 (+ 20us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:15.575841 (+ 16us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260619 02:42:15.575955 21227 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 02:42:15.580823 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 21098
I20260619 02:42:15.581049 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.67:39503
--local_ip_for_outbound_sockets=127.5.241.67
--tserver_master_addrs=127.5.241.126:45629
--webserver_port=35069
--webserver_interface=127.5.241.67
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab
--principal=kudu/127.5.241.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:42191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:42:15.662168 21230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:42:15.662380 21230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:42:15.662420 21230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:42:15.663637 21230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:42:15.663682 21230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:42:15.663710 21230 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.67
I20260619 02:42:15.665230 21230 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:42191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab
--principal=kudu/127.5.241.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.67:39503
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.241.67
--webserver_port=35069
--tserver_master_addrs=127.5.241.126:45629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21230
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.67
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:42:15.665489 21230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:42:15.665724 21230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:42:15.670750 21238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:15.670763 21236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:42:15.671000 21230 server_base.cc:1034] running on GCE node
W20260619 02:42:15.670763 21235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:42:15.671245 21230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:42:15.671518 21230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:42:15.672674 21230 hybrid_clock.cc:648] HybridClock initialized: now 1781836935672665 us; error 32 us; skew 500 ppm
Jun 19 02:42:15 dist-test-slave-zpbg krb5kdc[19797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836935, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:42:15.675133 21230 init.cc:377] Logged in from keytab as kudu/127.5.241.67@KRBTEST.COM (short username kudu)
I20260619 02:42:15.675575 21230 webserver.cc:460] Webserver started at http://127.5.241.67:35069/ using document root <none> and password file <none>
I20260619 02:42:15.675809 21230 fs_manager.cc:359] Metadata directory not provided
I20260619 02:42:15.675882 21230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:42:15.677258 21230 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:42:15.677955 21245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:15.678188 21230 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:42:15.678300 21230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-2/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-2/wal
uuid: "ca02d260fb4842dcb4121f3c88a64508"
format_stamp: "Formatted at 2026-06-19 02:42:14 on dist-test-slave-zpbg"
server_key: "bf1afc2bede4ed86072d0561befb329f"
server_key_iv: "c0dab91368041e3d6923351a0eaf8cc1"
server_key_version: "encryptionkey@0"
I20260619 02:42:15.678660 21230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:42:15.689287 21230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:42:15.691638 21230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:42:15.691799 21230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:42:15.692031 21230 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:42:15.692333 21230 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:42:15.692389 21230 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:15.692430 21230 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:42:15.692457 21230 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:15.698083 21230 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.67:39503
I20260619 02:42:15.698122 21358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.67:39503 every 8 connection(s)
I20260619 02:42:15.698477 21230 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
I20260619 02:42:15.699544 21361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:15.699166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.67:36125)
0619 02:42:15.699332 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:15.699402 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":74,"threads_started":1}
W20260619 02:42:15.700034 21359 heartbeater.cc:644] Failed to heartbeat to 127.5.241.126:45629 (0 consecutive failures): Network error: Failed to ping master at 127.5.241.126:45629: Client connection negotiation failed: client connection to 127.5.241.126:45629: connect: Connection refused (error 111)
I20260619 02:42:15.700348 21361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:15.700225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.67:37377)
0619 02:42:15.700300 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:15.700328 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260619 02:42:15.700634 21361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:15.700550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.67:38791)
0619 02:42:15.700597 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:15.700618 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260619 02:42:15.700742 21359 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 02:42:15.705803 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 21230
I20260619 02:42:16.514863 21363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:16.514536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.65:35459)
0619 02:42:16.514684 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:16.514724 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":63,"threads_started":1}
I20260619 02:42:16.576578 21364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:16.576262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.66:48611)
0619 02:42:16.576409 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:16.576449 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":68,"threads_started":1}
I20260619 02:42:16.701313 21365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:16.701050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.67:33995)
0619 02:42:16.701201 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:16.701240 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":71,"threads_started":1}
I20260619 02:42:16.717039 21366 raft_consensus.cc:482] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:42:16.717181 21366 raft_consensus.cc:504] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "816a43ca3f4842079542408ac9fde61f" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 46401 } } peers { permanent_uuid: "ca02d260fb4842dcb4121f3c88a64508" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 39503 } } peers { permanent_uuid: "00c9712fe62f4d23a46d6270a86101e9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46275 } }
I20260619 02:42:16.717628 21366 leader_election.cc:288] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 816a43ca3f4842079542408ac9fde61f (127.5.241.66:46401), ca02d260fb4842dcb4121f3c88a64508 (127.5.241.67:39503)
Jun 19 02:42:16 dist-test-slave-zpbg krb5kdc[19797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836935, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.65@KRBTEST.COM for kudu/127.5.241.67@KRBTEST.COM
Jun 19 02:42:16 dist-test-slave-zpbg krb5kdc[19797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836935, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.65@KRBTEST.COM for kudu/127.5.241.66@KRBTEST.COM
I20260619 02:42:16.728713 21363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:16.717837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.67:39503 (local address 127.5.241.65:48409)
0619 02:42:16.717896 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:16.717909 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:16.717974 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:16.719063 (+ 1089us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:16.719065 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:16.719427 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:42:16.719615 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:16.719620 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:16.720729 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:16.720730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:16.721681 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:16.721683 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:16.722314 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:16.722315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:16.722434 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:16.723054 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:42:16.723063 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:42:16.724654 (+ 1591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:42:16.726951 (+ 2297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:16.726957 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:16.726959 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:16.727256 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:16.727520 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:16.727521 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:16.727521 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:16.727656 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:16.728157 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:42:16.728164 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:42:16.728317 (+ 153us) client_negotiation.cc:769] Sending connection context
0619 02:42:16.728426 (+ 109us) client_negotiation.cc:240] Negotiation successful
0619 02:42:16.728614 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":52}
I20260619 02:42:16.728726 21369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:16.718058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:57699 (local address 127.5.241.66:46401)
0619 02:42:16.718221 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:16.718229 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:16.718256 (+ 27us) server_negotiation.cc:407] Connection header received
0619 02:42:16.718675 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:16.718677 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:16.718713 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:16.718884 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:42:16.719735 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:16.720583 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:16.721790 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:16.722198 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:16.722338 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:16.725238 (+ 2900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:42:16.725242 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:42:16.725242 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:42:16.725256 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:42:16.727107 (+ 1851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:16.727507 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:16.727510 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:16.727511 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:16.727631 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:16.727928 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:16.727928 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:16.727929 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:16.728173 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:42:16.728252 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:16.728477 (+ 225us) server_negotiation.cc:299] Negotiation successful
0619 02:42:16.728604 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":69,"threads_started":1}
I20260619 02:42:16.728751 21368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:16.717943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:48409 (local address 127.5.241.67:39503)
0619 02:42:16.718125 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:16.718137 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:16.718173 (+ 36us) server_negotiation.cc:407] Connection header received
0619 02:42:16.718663 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:16.718665 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:16.718707 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:16.718876 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:42:16.719757 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:16.720616 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:16.721782 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:16.722202 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:16.722339 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:16.724805 (+ 2466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:42:16.724810 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:42:16.724810 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:42:16.724824 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:42:16.726872 (+ 2048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:16.727332 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:16.727336 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:16.727336 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:16.727451 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:16.727746 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:16.727748 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:16.727748 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:16.728086 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:42:16.728150 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:16.728484 (+ 334us) server_negotiation.cc:299] Negotiation successful
0619 02:42:16.728631 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":57,"threads_started":1}
I20260619 02:42:16.728825 21367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:16.717865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.66:46401 (local address 127.5.241.65:57699)
0619 02:42:16.718017 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:16.718028 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:16.718089 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:16.719067 (+ 978us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:16.719068 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:16.719426 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:42:16.719615 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:16.719620 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:16.720698 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:16.720699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:16.721680 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:16.721683 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:16.722309 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:16.722310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:16.722434 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:16.723106 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:42:16.723111 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:42:16.725137 (+ 2026us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:42:16.727193 (+ 2056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:16.727194 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:16.727195 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:16.727411 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:16.727746 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:16.727747 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:16.727747 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:16.727856 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:16.728251 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:42:16.728252 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:42:16.728312 (+ 60us) client_negotiation.cc:769] Sending connection context
0619 02:42:16.728426 (+ 114us) client_negotiation.cc:240] Negotiation successful
0619 02:42:16.728697 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":179,"thread_start_us":74,"threads_started":1}
W20260619 02:42:16.730206 20981 leader_election.cc:341] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 816a43ca3f4842079542408ac9fde61f (127.5.241.66:46401): Not found: Tablet not found: 2bc943dcbdd147899aab858c990dfdd9
W20260619 02:42:16.730422 20979 leader_election.cc:341] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ca02d260fb4842dcb4121f3c88a64508 (127.5.241.67:39503): Not found: Tablet not found: 2bc943dcbdd147899aab858c990dfdd9
I20260619 02:42:16.730479 20979 leader_election.cc:302] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00c9712fe62f4d23a46d6270a86101e9; no voters: 816a43ca3f4842079542408ac9fde61f, ca02d260fb4842dcb4121f3c88a64508
I20260619 02:42:16.730577 21366 raft_consensus.cc:2726] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 02:42:17.515488 21370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:17.515278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.65:34343)
0619 02:42:17.515403 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:17.515442 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":58,"threads_started":1}
I20260619 02:42:17.577251 21371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:17.577065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.66:50133)
0619 02:42:17.577182 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:17.577214 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":56,"threads_started":1}
I20260619 02:42:17.701889 21372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:17.701679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.67:57703)
0619 02:42:17.701818 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:17.701851 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":57,"threads_started":1}
I20260619 02:42:18.516191 21373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:18.515966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.65:55571)
0619 02:42:18.516105 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:18.516142 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":60,"threads_started":1}
I20260619 02:42:18.577929 21374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:18.577722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.66:56749)
0619 02:42:18.577850 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:18.577886 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":65,"threads_started":1}
I20260619 02:42:18.702556 21375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:18.702339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.67:41999)
0619 02:42:18.702479 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:18.702516 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":80,"threads_started":1}
I20260619 02:42:18.844219 21376 raft_consensus.cc:482] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:42:18.844318 21376 raft_consensus.cc:504] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "816a43ca3f4842079542408ac9fde61f" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 46401 } } peers { permanent_uuid: "ca02d260fb4842dcb4121f3c88a64508" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 39503 } } peers { permanent_uuid: "00c9712fe62f4d23a46d6270a86101e9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46275 } }
I20260619 02:42:18.844532 21376 leader_election.cc:288] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 816a43ca3f4842079542408ac9fde61f (127.5.241.66:46401), ca02d260fb4842dcb4121f3c88a64508 (127.5.241.67:39503)
W20260619 02:42:18.845036 20981 leader_election.cc:341] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 816a43ca3f4842079542408ac9fde61f (127.5.241.66:46401): Not found: Tablet not found: 2bc943dcbdd147899aab858c990dfdd9
W20260619 02:42:18.845124 20979 leader_election.cc:341] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ca02d260fb4842dcb4121f3c88a64508 (127.5.241.67:39503): Not found: Tablet not found: 2bc943dcbdd147899aab858c990dfdd9
I20260619 02:42:18.845216 20979 leader_election.cc:302] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00c9712fe62f4d23a46d6270a86101e9; no voters: 816a43ca3f4842079542408ac9fde61f, ca02d260fb4842dcb4121f3c88a64508
I20260619 02:42:18.845288 21376 raft_consensus.cc:2726] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 02:42:19.516952 21377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:19.516742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.65:33729)
0619 02:42:19.516868 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:19.516905 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":66,"threads_started":1}
I20260619 02:42:19.578603 21378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:19.578351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.66:48165)
0619 02:42:19.578485 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:19.578544 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":91,"threads_started":1}
I20260619 02:42:19.703410 21379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:19.703142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.67:57215)
0619 02:42:19.703296 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:19.703346 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":70,"threads_started":1}
I20260619 02:42:20.517870 21380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:20.517549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.65:39757)
0619 02:42:20.517758 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:20.517808 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":65,"threads_started":1}
I20260619 02:42:20.579430 21381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:20.579143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.66:42873)
0619 02:42:20.579319 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:20.579373 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":62,"threads_started":1}
I20260619 02:42:20.616079 21382 raft_consensus.cc:482] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:42:20.616194 21382 raft_consensus.cc:504] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "816a43ca3f4842079542408ac9fde61f" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 46401 } } peers { permanent_uuid: "ca02d260fb4842dcb4121f3c88a64508" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 39503 } } peers { permanent_uuid: "00c9712fe62f4d23a46d6270a86101e9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46275 } }
I20260619 02:42:20.616421 21382 leader_election.cc:288] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 816a43ca3f4842079542408ac9fde61f (127.5.241.66:46401), ca02d260fb4842dcb4121f3c88a64508 (127.5.241.67:39503)
W20260619 02:42:20.616796 20981 leader_election.cc:341] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 816a43ca3f4842079542408ac9fde61f (127.5.241.66:46401): Not found: Tablet not found: 2bc943dcbdd147899aab858c990dfdd9
W20260619 02:42:20.616876 20979 leader_election.cc:341] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ca02d260fb4842dcb4121f3c88a64508 (127.5.241.67:39503): Not found: Tablet not found: 2bc943dcbdd147899aab858c990dfdd9
I20260619 02:42:20.616952 20979 leader_election.cc:302] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00c9712fe62f4d23a46d6270a86101e9; no voters: 816a43ca3f4842079542408ac9fde61f, ca02d260fb4842dcb4121f3c88a64508
I20260619 02:42:20.617044 21382 raft_consensus.cc:2726] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 02:42:20.704144 21383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:20.703945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.67:58783)
0619 02:42:20.704061 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:20.704102 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":57,"threads_started":1}
I20260619 02:42:21.518642 21384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:21.518386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.65:58795)
0619 02:42:21.518543 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:21.518596 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":69,"threads_started":1}
I20260619 02:42:21.580185 21385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:21.579948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.66:48621)
0619 02:42:21.580097 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:21.580140 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":80,"threads_started":1}
I20260619 02:42:21.704950 21386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:21.704712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.67:55193)
0619 02:42:21.704863 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:21.704903 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":61,"threads_started":1}
I20260619 02:42:22.519381 21387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:22.519149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.65:36007)
0619 02:42:22.519295 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:22.519334 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":76,"threads_started":1}
I20260619 02:42:22.580884 21388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:22.580649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.66:57801)
0619 02:42:22.580804 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:22.580841 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":78,"threads_started":1}
I20260619 02:42:22.705592 21389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:22.705400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.67:44073)
0619 02:42:22.705516 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:22.705549 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":58,"threads_started":1}
I20260619 02:42:23.520236 21390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:23.520031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.65:37773)
0619 02:42:23.520150 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:23.520189 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":61,"threads_started":1}
I20260619 02:42:23.581609 21391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:23.581400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.66:47571)
0619 02:42:23.581528 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:23.581566 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":64,"threads_started":1}
I20260619 02:42:23.706321 21392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:23.706087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.67:33717)
0619 02:42:23.706240 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:23.706275 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":86,"threads_started":1}
I20260619 02:42:24.520947 21393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:24.520740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.65:36829)
0619 02:42:24.520863 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:24.520901 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":61,"threads_started":1}
I20260619 02:42:24.582297 21394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:24.582068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.66:59817)
0619 02:42:24.582212 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:24.582255 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":58,"threads_started":1}
I20260619 02:42:24.658598 21395 raft_consensus.cc:482] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:42:24.658695 21395 raft_consensus.cc:504] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "816a43ca3f4842079542408ac9fde61f" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 46401 } } peers { permanent_uuid: "ca02d260fb4842dcb4121f3c88a64508" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 39503 } } peers { permanent_uuid: "00c9712fe62f4d23a46d6270a86101e9" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46275 } }
I20260619 02:42:24.658937 21395 leader_election.cc:288] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 816a43ca3f4842079542408ac9fde61f (127.5.241.66:46401), ca02d260fb4842dcb4121f3c88a64508 (127.5.241.67:39503)
W20260619 02:42:24.659372 20979 leader_election.cc:341] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ca02d260fb4842dcb4121f3c88a64508 (127.5.241.67:39503): Not found: Tablet not found: 2bc943dcbdd147899aab858c990dfdd9
W20260619 02:42:24.659454 20981 leader_election.cc:341] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 816a43ca3f4842079542408ac9fde61f (127.5.241.66:46401): Not found: Tablet not found: 2bc943dcbdd147899aab858c990dfdd9
I20260619 02:42:24.659571 20981 leader_election.cc:302] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00c9712fe62f4d23a46d6270a86101e9; no voters: 816a43ca3f4842079542408ac9fde61f, ca02d260fb4842dcb4121f3c88a64508
I20260619 02:42:24.659690 21395 raft_consensus.cc:2726] T 2bc943dcbdd147899aab858c990dfdd9 P 00c9712fe62f4d23a46d6270a86101e9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 02:42:24.707103 21396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:24.706863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.67:43271)
0619 02:42:24.707010 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:24.707055 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":55,"threads_started":1}
I20260619 02:42:25.521739 21397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:25.521498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.65:44905)
0619 02:42:25.521659 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:25.521693 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":107,"threads_started":1}
I20260619 02:42:25.583129 21398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:25.582907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.66:36287)
0619 02:42:25.583045 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:25.583085 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":75,"threads_started":1}
I20260619 02:42:25.707955 21399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:25.707715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.67:58713)
0619 02:42:25.707865 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:25.707909 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":62,"threads_started":1}
I20260619 02:42:26.522576 21400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:26.522322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.65:51223)
0619 02:42:26.522480 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:26.522526 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":77,"threads_started":1}
I20260619 02:42:26.583796 21401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:26.583579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.66:59725)
0619 02:42:26.583712 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:26.583751 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":63,"threads_started":1}
I20260619 02:42:26.708861 21402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:26.708490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.67:48203)
0619 02:42:26.708773 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:26.708815 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":226,"threads_started":1}
I20260619 02:42:27.523403 21403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:27.523184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.65:59585)
0619 02:42:27.523320 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:27.523356 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":77,"threads_started":1}
I20260619 02:42:27.584471 21404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:27.584269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.66:35691)
0619 02:42:27.584390 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:27.584428 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":59,"threads_started":1}
I20260619 02:42:27.709554 21405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:27.709340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.67:41835)
0619 02:42:27.709475 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:27.709511 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":72,"threads_started":1}
I20260619 02:42:28.524063 21406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:28.523845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.65:41669)
0619 02:42:28.523980 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:28.524018 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":62,"threads_started":1}
I20260619 02:42:28.585139 21407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:28.584939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.66:50383)
0619 02:42:28.585062 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:28.585095 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":59,"threads_started":1}
I20260619 02:42:28.710199 21408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:28.710006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.67:53165)
0619 02:42:28.710122 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:28.710155 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":65,"threads_started":1}
I20260619 02:42:29.524791 21409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:29.524561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.65:39807)
0619 02:42:29.524690 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:29.524732 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":63,"threads_started":1}
I20260619 02:42:29.585861 21410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:29.585633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.66:51147)
0619 02:42:29.585772 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:29.585815 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":61,"threads_started":1}
I20260619 02:42:29.710958 21411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:29.710672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.67:36381)
0619 02:42:29.710850 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:29.710888 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":61,"threads_started":1}
I20260619 02:42:30.525550 21412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:30.525335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.65:60045)
0619 02:42:30.525467 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:30.525505 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":73,"threads_started":1}
I20260619 02:42:30.586707 21413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:30.586400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.66:59489)
0619 02:42:30.586593 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:30.586655 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":84,"threads_started":1}
I20260619 02:42:30.711808 21414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:30.711593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45629 (local address 127.5.241.67:41611)
0619 02:42:30.711727 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:30.711765 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":59,"threads_started":1}
I20260619 02:42:30.715366 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 20934
I20260619 02:42:30.720297 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 21098
I20260619 02:42:30.724924 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 21230
2026-06-19T02:42:30Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18233 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:42:30 dist-test-slave-zpbg krb5kdc[21417](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:42:30 dist-test-slave-zpbg krb5kdc[21417](info): set up 2 sockets
Jun 19 02:42:30 dist-test-slave-zpbg krb5kdc[21417](info): commencing operation
krb5kdc: starting...
W20260619 02:42:31.770870 6085 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Jun 19 02:42:31 dist-test-slave-zpbg krb5kdc[21417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836951, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:42:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:42:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.241.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
I20260619 02:42:31.890292 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:39127
--webserver_interface=127.5.241.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:44479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:39127
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:42:31.970284 21430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:42:31.970489 21430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:42:31.970510 21430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:42:31.971657 21430 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 02:42:31.971695 21430 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 02:42:31.971709 21430 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 02:42:31.971719 21430 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:42:31.971732 21430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:42:31.971745 21430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:42:31.971757 21430 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:42:31.971769 21430 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:42:31.973145 21430 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:44479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:39127
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:39127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.241.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21430
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:42:31.973337 21430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:42:31.973495 21430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:42:31.978204 21435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:42:31.978266 21430 server_base.cc:1034] running on GCE node
W20260619 02:42:31.978191 21436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:31.978194 21438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:42:31.978657 21430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:42:31.978981 21430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:42:31.980131 21430 hybrid_clock.cc:648] HybridClock initialized: now 1781836951980118 us; error 27 us; skew 500 ppm
Jun 19 02:42:31 dist-test-slave-zpbg krb5kdc[21417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836951, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:42:31.986562 21430 init.cc:377] Logged in from keytab as kudu/127.5.241.126@KRBTEST.COM (short username kudu)
I20260619 02:42:31.987040 21430 webserver.cc:460] Webserver started at http://127.5.241.126:34563/ using document root <none> and password file <none>
I20260619 02:42:31.987279 21430 fs_manager.cc:359] Metadata directory not provided
I20260619 02:42:31.987325 21430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:42:31.987468 21430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:42:31.988528 21430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/master-0/data/instance:
uuid: "6c315d868f78434a8e1c444c0279e190"
format_stamp: "Formatted at 2026-06-19 02:42:31 on dist-test-slave-zpbg"
server_key: "db9e53fe736cf602ca04bf799dbfb167"
server_key_iv: "135d364f0237f5781bdfbfb89800aee9"
server_key_version: "encryptionkey@0"
I20260619 02:42:31.988889 21430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/master-0/wal/instance:
uuid: "6c315d868f78434a8e1c444c0279e190"
format_stamp: "Formatted at 2026-06-19 02:42:31 on dist-test-slave-zpbg"
server_key: "db9e53fe736cf602ca04bf799dbfb167"
server_key_iv: "135d364f0237f5781bdfbfb89800aee9"
server_key_version: "encryptionkey@0"
I20260619 02:42:31.990252 21430 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:42:31.991030 21445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:31.991201 21430 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:42:31.991289 21430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "6c315d868f78434a8e1c444c0279e190"
format_stamp: "Formatted at 2026-06-19 02:42:31 on dist-test-slave-zpbg"
server_key: "db9e53fe736cf602ca04bf799dbfb167"
server_key_iv: "135d364f0237f5781bdfbfb89800aee9"
server_key_version: "encryptionkey@0"
I20260619 02:42:31.991349 21430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:42:32.002697 21430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:42:32.004662 21430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:42:32.004767 21430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:42:32.008543 21430 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.126:39127
I20260619 02:42:32.008589 21497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.126:39127 every 8 connection(s)
I20260619 02:42:32.008911 21430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:42:32.009745 21498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:42:32.011896 21498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190: Bootstrap starting.
I20260619 02:42:32.012465 21498 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190: Neither blocks nor log segments found. Creating new log.
I20260619 02:42:32.012749 21498 log.cc:824] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190: Log is configured to *not* fsync() on all Append() calls
I20260619 02:42:32.013391 21498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190: No bootstrap required, opened a new log
I20260619 02:42:32.014940 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 21430
I20260619 02:42:32.015054 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/master-0/wal/instance
I20260619 02:42:32.015008 21498 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c315d868f78434a8e1c444c0279e190" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 39127 } }
I20260619 02:42:32.015204 21498 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:42:32.015255 21498 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c315d868f78434a8e1c444c0279e190, State: Initialized, Role: FOLLOWER
I20260619 02:42:32.015316 6085 external_mini_cluster.cc:1428] Setting key f1b479d45946dc28e02e9553b7959b4d
I20260619 02:42:32.015412 21498 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c315d868f78434a8e1c444c0279e190" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 39127 } }
I20260619 02:42:32.015498 21498 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:42:32.015563 21498 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:42:32.015622 21498 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:42:32.016723 21498 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c315d868f78434a8e1c444c0279e190" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 39127 } }
I20260619 02:42:32.016844 21498 leader_election.cc:302] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c315d868f78434a8e1c444c0279e190; no voters:
I20260619 02:42:32.017004 21498 leader_election.cc:288] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:42:32.017030 21503 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:42:32.017135 21503 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190 [term 1 LEADER]: Becoming Leader. State: Replica: 6c315d868f78434a8e1c444c0279e190, State: Running, Role: LEADER
I20260619 02:42:32.017267 21503 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c315d868f78434a8e1c444c0279e190" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 39127 } }
I20260619 02:42:32.017277 21498 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:42:32.017705 21504 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c315d868f78434a8e1c444c0279e190. Latest consensus state: current_term: 1 leader_uuid: "6c315d868f78434a8e1c444c0279e190" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c315d868f78434a8e1c444c0279e190" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 39127 } } }
I20260619 02:42:32.017799 21504 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190 [sys.catalog]: This master's current role is: LEADER
I20260619 02:42:32.018028 21505 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c315d868f78434a8e1c444c0279e190" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c315d868f78434a8e1c444c0279e190" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 39127 } } }
I20260619 02:42:32.018146 21505 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190 [sys.catalog]: This master's current role is: LEADER
I20260619 02:42:32.018299 21508 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:42:32.018834 21508 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:42:32.020324 21508 catalog_manager.cc:1347] Generated new cluster ID: 21a3f16bf7204b8e8b7a4af68209b14f
I20260619 02:42:32.020380 21508 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 02:42:32 dist-test-slave-zpbg krb5kdc[21417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836951, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:42:32.026973 21502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:32.015823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43058 (local address 127.5.241.126:39127)
0619 02:42:32.016036 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:32.016045 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:32.016085 (+ 40us) server_negotiation.cc:407] Connection header received
0619 02:42:32.016220 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:32.016228 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:32.016292 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:32.016465 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:42:32.017248 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:32.018743 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:32.019771 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:32.020380 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:32.020481 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:32.023403 (+ 2922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:42:32.023407 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:42:32.023407 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:42:32.023418 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:42:32.025365 (+ 1947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:32.025761 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:32.025764 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:32.025764 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:32.025866 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:32.026072 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:32.026073 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:32.026073 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:32.026351 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:42:32.026473 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:32.026608 (+ 135us) server_negotiation.cc:299] Negotiation successful
0619 02:42:32.026729 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":96,"threads_started":1}
I20260619 02:42:32.043186 21508 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:42:32.043820 21508 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:42:32.046473 21508 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190: Generated new TSK 0
I20260619 02:42:32.046663 21508 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.241.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab.
Entry for principal kudu/127.5.241.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab.
I20260619 02:42:32.078389 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.65:0
--local_ip_for_outbound_sockets=127.5.241.65
--webserver_interface=127.5.241.65
--webserver_port=0
--tserver_master_addrs=127.5.241.126:39127
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab
--principal=kudu/127.5.241.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:44479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:42:32.160626 21524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:42:32.160854 21524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:42:32.160890 21524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:42:32.162003 21524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:42:32.162065 21524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:42:32.162110 21524 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.65
I20260619 02:42:32.163564 21524 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:44479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab
--principal=kudu/127.5.241.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.241.65
--webserver_port=0
--tserver_master_addrs=127.5.241.126:39127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.65
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:42:32.163823 21524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:42:32.164043 21524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:42:32.169154 21524 server_base.cc:1034] running on GCE node
W20260619 02:42:32.169124 21529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:32.169124 21532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:32.169124 21530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:42:32.169595 21524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:42:32.169878 21524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:42:32.171046 21524 hybrid_clock.cc:648] HybridClock initialized: now 1781836952171038 us; error 34 us; skew 500 ppm
Jun 19 02:42:32 dist-test-slave-zpbg krb5kdc[21417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836952, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:42:32.173650 21524 init.cc:377] Logged in from keytab as kudu/127.5.241.65@KRBTEST.COM (short username kudu)
I20260619 02:42:32.174147 21524 webserver.cc:460] Webserver started at http://127.5.241.65:35705/ using document root <none> and password file <none>
I20260619 02:42:32.174384 21524 fs_manager.cc:359] Metadata directory not provided
I20260619 02:42:32.174454 21524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:42:32.174604 21524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:42:32.175683 21524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-0/data/instance:
uuid: "1c084879cb83420590952a75fa02ff81"
format_stamp: "Formatted at 2026-06-19 02:42:32 on dist-test-slave-zpbg"
server_key: "de1ee857088991bde4e566a46099bd1f"
server_key_iv: "bed7176c6ba9e35235a3f6062a0957a7"
server_key_version: "encryptionkey@0"
I20260619 02:42:32.176033 21524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-0/wal/instance:
uuid: "1c084879cb83420590952a75fa02ff81"
format_stamp: "Formatted at 2026-06-19 02:42:32 on dist-test-slave-zpbg"
server_key: "de1ee857088991bde4e566a46099bd1f"
server_key_iv: "bed7176c6ba9e35235a3f6062a0957a7"
server_key_version: "encryptionkey@0"
I20260619 02:42:32.177357 21524 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:42:32.178105 21539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:32.178287 21524 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:42:32.178376 21524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-0/wal
uuid: "1c084879cb83420590952a75fa02ff81"
format_stamp: "Formatted at 2026-06-19 02:42:32 on dist-test-slave-zpbg"
server_key: "de1ee857088991bde4e566a46099bd1f"
server_key_iv: "bed7176c6ba9e35235a3f6062a0957a7"
server_key_version: "encryptionkey@0"
I20260619 02:42:32.178455 21524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:42:32.188223 21524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:42:32.190469 21524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:42:32.190629 21524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:42:32.190918 21524 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:42:32.191227 21524 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:42:32.191277 21524 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:32.191319 21524 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:42:32.191349 21524 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:32.197274 21524 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.65:39547
I20260619 02:42:32.197359 21652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.65:39547 every 8 connection(s)
I20260619 02:42:32.197679 21524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
I20260619 02:42:32.203387 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 21524
I20260619 02:42:32.203482 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-0/wal/instance
Jun 19 02:42:32 dist-test-slave-zpbg krb5kdc[21417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836952, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.65@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:42:32.203677 6085 external_mini_cluster.cc:1428] Setting key f434c27d22a3bb97cecf4c8e4ab39735
I20260619 02:42:32.207110 21502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:32.198345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:48347 (local address 127.5.241.126:39127)
0619 02:42:32.198410 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:32.198412 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:32.198619 (+ 207us) server_negotiation.cc:407] Connection header received
0619 02:42:32.199147 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:32.199148 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:32.199159 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:32.199205 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:42:32.200113 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:32.200618 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:32.201571 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:32.201757 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:32.201830 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:32.204181 (+ 2351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:42:32.204183 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:42:32.204183 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:42:32.204191 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:42:32.205631 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:32.206043 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:32.206045 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:32.206046 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:32.206111 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:32.206337 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:32.206338 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:32.206339 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:32.206484 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:42:32.206561 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:32.206931 (+ 370us) server_negotiation.cc:299] Negotiation successful
0619 02:42:32.207019 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 02:42:32.207309 21655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:32.198343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.65:48347)
0619 02:42:32.198506 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:32.198539 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:32.199064 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:32.199263 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:32.199266 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:32.199835 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:42:32.200024 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:32.200028 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:32.200709 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:32.200709 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:32.201503 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:32.201504 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:32.201864 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:32.201865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:32.201974 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:32.202530 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:42:32.202538 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:42:32.204083 (+ 1545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:42:32.205735 (+ 1652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:32.205742 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:32.205745 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:32.205972 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:32.206183 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:32.206183 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:32.206184 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:32.206282 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:32.206572 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:42:32.206578 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:42:32.206695 (+ 117us) client_negotiation.cc:769] Sending connection context
0619 02:42:32.206901 (+ 206us) client_negotiation.cc:240] Negotiation successful
0619 02:42:32.207063 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":93,"threads_started":1}
I20260619 02:42:32.207918 21653 heartbeater.cc:347] Connected to a master server at 127.5.241.126:39127
I20260619 02:42:32.208035 21653 heartbeater.cc:464] Registering TS with master...
I20260619 02:42:32.208221 21653 heartbeater.cc:511] Master 127.5.241.126:39127 requested a full tablet report, sending...
I20260619 02:42:32.208724 21462 ts_manager.cc:194] Registered new tserver with Master: 1c084879cb83420590952a75fa02ff81 (127.5.241.65:39547)
I20260619 02:42:32.209332 21462 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.65@KRBTEST.COM'} at 127.5.241.65:48347
WARNING: no policy specified for kudu/127.5.241.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab.
Entry for principal kudu/127.5.241.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab.
I20260619 02:42:32.228832 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.66:0
--local_ip_for_outbound_sockets=127.5.241.66
--webserver_interface=127.5.241.66
--webserver_port=0
--tserver_master_addrs=127.5.241.126:39127
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab
--principal=kudu/127.5.241.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:44479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:42:32.313591 21658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:42:32.313875 21658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:42:32.313920 21658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:42:32.315598 21658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:42:32.315673 21658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:42:32.315722 21658 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.66
I20260619 02:42:32.317353 21658 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:44479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab
--principal=kudu/127.5.241.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.241.66
--webserver_port=0
--tserver_master_addrs=127.5.241.126:39127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21658
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.66
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:42:32.317631 21658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:42:32.317868 21658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:42:32.323144 21664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:42:32.323223 21658 server_base.cc:1034] running on GCE node
W20260619 02:42:32.323165 21663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:32.323258 21666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:42:32.323554 21658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:42:32.323844 21658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:42:32.325016 21658 hybrid_clock.cc:648] HybridClock initialized: now 1781836952325003 us; error 33 us; skew 500 ppm
Jun 19 02:42:32 dist-test-slave-zpbg krb5kdc[21417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836952, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:42:32.327757 21658 init.cc:377] Logged in from keytab as kudu/127.5.241.66@KRBTEST.COM (short username kudu)
I20260619 02:42:32.328249 21658 webserver.cc:460] Webserver started at http://127.5.241.66:44697/ using document root <none> and password file <none>
I20260619 02:42:32.328501 21658 fs_manager.cc:359] Metadata directory not provided
I20260619 02:42:32.328572 21658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:42:32.328729 21658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:42:32.329835 21658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-1/data/instance:
uuid: "1e6244cc32344d32ba646eb8c9a6f965"
format_stamp: "Formatted at 2026-06-19 02:42:32 on dist-test-slave-zpbg"
server_key: "3d080433e7293677d7af8ae265138acf"
server_key_iv: "8ad93a6546cc48599734ef16cf090ab5"
server_key_version: "encryptionkey@0"
I20260619 02:42:32.330229 21658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-1/wal/instance:
uuid: "1e6244cc32344d32ba646eb8c9a6f965"
format_stamp: "Formatted at 2026-06-19 02:42:32 on dist-test-slave-zpbg"
server_key: "3d080433e7293677d7af8ae265138acf"
server_key_iv: "8ad93a6546cc48599734ef16cf090ab5"
server_key_version: "encryptionkey@0"
I20260619 02:42:32.331774 21658 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:42:32.332631 21673 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:32.332824 21658 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:42:32.332930 21658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-1/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-1/wal
uuid: "1e6244cc32344d32ba646eb8c9a6f965"
format_stamp: "Formatted at 2026-06-19 02:42:32 on dist-test-slave-zpbg"
server_key: "3d080433e7293677d7af8ae265138acf"
server_key_iv: "8ad93a6546cc48599734ef16cf090ab5"
server_key_version: "encryptionkey@0"
I20260619 02:42:32.333009 21658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:42:32.342296 21658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:42:32.344656 21658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:42:32.344808 21658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:42:32.345005 21658 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:42:32.345278 21658 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:42:32.345314 21658 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:32.345340 21658 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:42:32.345355 21658 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:32.351716 21658 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.66:36607
I20260619 02:42:32.351809 21786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.66:36607 every 8 connection(s)
I20260619 02:42:32.352138 21658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
I20260619 02:42:32.353780 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 21658
I20260619 02:42:32.353897 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-1/wal/instance
I20260619 02:42:32.354113 6085 external_mini_cluster.cc:1428] Setting key 17222e19cd031c5dfd85a0c84f39a0e5
Jun 19 02:42:32 dist-test-slave-zpbg krb5kdc[21417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836952, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.66@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.241.67@KRBTEST.COM; defaulting to no policy
I20260619 02:42:32.362807 21502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:32.352947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:59045 (local address 127.5.241.126:39127)
0619 02:42:32.353035 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:32.353037 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:32.353231 (+ 194us) server_negotiation.cc:407] Connection header received
0619 02:42:32.353958 (+ 727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:32.353961 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:32.353976 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:32.354058 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:42:32.355421 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:32.356006 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:32.357148 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:32.357366 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:32.357451 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:32.359718 (+ 2267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:42:32.359720 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:42:32.359720 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:42:32.359728 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:42:32.361263 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:32.361720 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:32.361721 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:32.361722 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:32.361790 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:32.362054 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:32.362055 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:32.362056 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:32.362185 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:42:32.362257 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:32.362580 (+ 323us) server_negotiation.cc:299] Negotiation successful
0619 02:42:32.362681 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 02:42:32.362991 21789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:32.352913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.66:59045)
0619 02:42:32.353101 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:32.353137 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:32.353815 (+ 678us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:32.354128 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:32.354133 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:32.355008 (+ 875us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:42:32.355304 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:32.355310 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:32.356112 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:32.356113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:32.357054 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:32.357056 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:32.357461 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:32.357462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:32.357602 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:32.358167 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:42:32.358180 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:42:32.359620 (+ 1440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:42:32.361374 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:32.361382 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:32.361385 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:32.361632 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:32.361858 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:32.361859 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:32.361859 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:32.361978 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:32.362249 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:42:32.362256 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:42:32.362437 (+ 181us) client_negotiation.cc:769] Sending connection context
0619 02:42:32.362559 (+ 122us) client_negotiation.cc:240] Negotiation successful
0619 02:42:32.362730 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":91,"threads_started":1}
I20260619 02:42:32.363499 21787 heartbeater.cc:347] Connected to a master server at 127.5.241.126:39127
I20260619 02:42:32.363606 21787 heartbeater.cc:464] Registering TS with master...
I20260619 02:42:32.363785 21787 heartbeater.cc:511] Master 127.5.241.126:39127 requested a full tablet report, sending...
I20260619 02:42:32.364141 21462 ts_manager.cc:194] Registered new tserver with Master: 1e6244cc32344d32ba646eb8c9a6f965 (127.5.241.66:36607)
I20260619 02:42:32.364512 21462 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.66@KRBTEST.COM'} at 127.5.241.66:59045
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab.
Entry for principal kudu/127.5.241.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab.
I20260619 02:42:32.381852 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.67:0
--local_ip_for_outbound_sockets=127.5.241.67
--webserver_interface=127.5.241.67
--webserver_port=0
--tserver_master_addrs=127.5.241.126:39127
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab
--principal=kudu/127.5.241.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:44479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:42:32.464068 21792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:42:32.464288 21792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:42:32.464314 21792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:42:32.465497 21792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:42:32.465548 21792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:42:32.465591 21792 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.67
I20260619 02:42:32.467228 21792 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:44479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab
--principal=kudu/127.5.241.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.241.67
--webserver_port=0
--tserver_master_addrs=127.5.241.126:39127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21792
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.67
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:42:32.467422 21792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:42:32.467604 21792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:42:32.472512 21797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:32.472520 21800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:32.472558 21798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:42:32.472764 21792 server_base.cc:1034] running on GCE node
I20260619 02:42:32.472976 21792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:42:32.473222 21792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:42:32.474448 21792 hybrid_clock.cc:648] HybridClock initialized: now 1781836952474431 us; error 27 us; skew 500 ppm
Jun 19 02:42:32 dist-test-slave-zpbg krb5kdc[21417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836952, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:42:32.477128 21792 init.cc:377] Logged in from keytab as kudu/127.5.241.67@KRBTEST.COM (short username kudu)
I20260619 02:42:32.477608 21792 webserver.cc:460] Webserver started at http://127.5.241.67:35923/ using document root <none> and password file <none>
I20260619 02:42:32.477838 21792 fs_manager.cc:359] Metadata directory not provided
I20260619 02:42:32.477905 21792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:42:32.478039 21792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:42:32.479180 21792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-2/data/instance:
uuid: "51b5ade6b85d4ab2976d15aea366ef09"
format_stamp: "Formatted at 2026-06-19 02:42:32 on dist-test-slave-zpbg"
server_key: "78c5584faf61b792b41bd0209d810b76"
server_key_iv: "855742e05241d138923a434f6ef22ed2"
server_key_version: "encryptionkey@0"
I20260619 02:42:32.479537 21792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-2/wal/instance:
uuid: "51b5ade6b85d4ab2976d15aea366ef09"
format_stamp: "Formatted at 2026-06-19 02:42:32 on dist-test-slave-zpbg"
server_key: "78c5584faf61b792b41bd0209d810b76"
server_key_iv: "855742e05241d138923a434f6ef22ed2"
server_key_version: "encryptionkey@0"
I20260619 02:42:32.480939 21792 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:42:32.481719 21807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:32.481884 21792 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:42:32.481959 21792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-2/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-2/wal
uuid: "51b5ade6b85d4ab2976d15aea366ef09"
format_stamp: "Formatted at 2026-06-19 02:42:32 on dist-test-slave-zpbg"
server_key: "78c5584faf61b792b41bd0209d810b76"
server_key_iv: "855742e05241d138923a434f6ef22ed2"
server_key_version: "encryptionkey@0"
I20260619 02:42:32.482013 21792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:42:32.492951 21792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:42:32.495340 21792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:42:32.495460 21792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:42:32.495661 21792 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:42:32.495941 21792 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:42:32.495971 21792 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:32.495994 21792 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:42:32.496007 21792 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:32.502017 21792 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.67:38187
I20260619 02:42:32.502106 21920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.67:38187 every 8 connection(s)
I20260619 02:42:32.502419 21792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
I20260619 02:42:32.507025 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 21792
I20260619 02:42:32.507146 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-2/wal/instance
I20260619 02:42:32.507416 6085 external_mini_cluster.cc:1428] Setting key 52ef7265854b9db89e31fa0ab7ab215c
Jun 19 02:42:32 dist-test-slave-zpbg krb5kdc[21417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836952, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.67@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:42:32.511750 21502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:32.503198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:35983 (local address 127.5.241.126:39127)
0619 02:42:32.503279 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:32.503281 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:32.503493 (+ 212us) server_negotiation.cc:407] Connection header received
0619 02:42:32.503948 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:32.503949 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:32.503960 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:32.504027 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:42:32.505012 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:32.505516 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:32.506567 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:32.506826 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:32.506896 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:32.509251 (+ 2355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:42:32.509252 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:42:32.509252 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:42:32.509259 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:42:32.510431 (+ 1172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:32.510861 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:32.510863 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:32.510863 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:32.510924 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:32.511140 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:32.511141 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:32.511141 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:32.511260 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:42:32.511321 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:32.511582 (+ 261us) server_negotiation.cc:299] Negotiation successful
0619 02:42:32.511658 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 02:42:32.511816 21923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:32.503168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.67:35983)
0619 02:42:32.503342 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:32.503384 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:32.503866 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:32.504097 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:32.504101 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:32.504702 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:42:32.504905 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:32.504910 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:32.505618 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:32.505619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:32.506483 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:32.506486 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:32.506919 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:32.506920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:32.507074 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:32.507718 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:42:32.507727 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:42:32.509163 (+ 1436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:42:32.510499 (+ 1336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:32.510505 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:32.510507 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:32.510723 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:32.510986 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:32.510987 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:32.510987 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:32.511087 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:32.511318 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:42:32.511323 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:42:32.511437 (+ 114us) client_negotiation.cc:769] Sending connection context
0619 02:42:32.511520 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 02:42:32.511642 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":99,"threads_started":1}
I20260619 02:42:32.512346 21921 heartbeater.cc:347] Connected to a master server at 127.5.241.126:39127
I20260619 02:42:32.512452 21921 heartbeater.cc:464] Registering TS with master...
I20260619 02:42:32.512636 21921 heartbeater.cc:511] Master 127.5.241.126:39127 requested a full tablet report, sending...
I20260619 02:42:32.513053 21462 ts_manager.cc:194] Registered new tserver with Master: 51b5ade6b85d4ab2976d15aea366ef09 (127.5.241.67:38187)
I20260619 02:42:32.513511 21462 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.67@KRBTEST.COM'} at 127.5.241.67:35983
I20260619 02:42:32.519407 6085 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:42:32.526492 21502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:32.520801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43074 (local address 127.5.241.126:39127)
0619 02:42:32.520858 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:32.520860 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:32.520962 (+ 102us) server_negotiation.cc:407] Connection header received
0619 02:42:32.521013 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:32.521014 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:32.521022 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:32.521078 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:42:32.521594 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:32.522066 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:32.522855 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:32.523031 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:32.523082 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:32.524194 (+ 1112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:42:32.524196 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:42:32.524196 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:42:32.524202 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:42:32.525435 (+ 1233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:32.525809 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:32.525810 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:32.525810 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:32.525859 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:32.526022 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:32.526022 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:32.526022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:32.526151 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:42:32.526223 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:32.526373 (+ 150us) server_negotiation.cc:299] Negotiation successful
0619 02:42:32.526430 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 02:42:32.528124 21462 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43074:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:42:32.529703 21462 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:42:32.536137 21931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:32.531906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48852 (local address 127.5.241.65:39547)
0619 02:42:32.532107 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:32.532113 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:32.532216 (+ 103us) server_negotiation.cc:407] Connection header received
0619 02:42:32.532316 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:32.532318 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:32.532333 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:32.532417 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:42:32.532564 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:32.533627 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:32.535069 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:32.535704 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:32.535800 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:32.535857 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:32.536016 (+ 159us) server_negotiation.cc:299] Negotiation successful
0619 02:42:32.536057 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":79,"threads_started":1}
I20260619 02:42:32.536137 21929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:32.531803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:39547 (local address 127.0.0.1:48852)
0619 02:42:32.532116 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:32.532154 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:32.532236 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:32.532433 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:32.532436 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:32.532438 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:42:32.532507 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:32.532508 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:32.533731 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:32.533732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:32.534975 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:32.534978 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:32.535804 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:32.535805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:32.535891 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:32.535904 (+ 13us) client_negotiation.cc:769] Sending connection context
0619 02:42:32.535997 (+ 93us) client_negotiation.cc:240] Negotiation successful
0619 02:42:32.536042 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"mutex_wait_us":41,"thread_start_us":52,"threads_started":1}
I20260619 02:42:32.536140 21930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:32.531806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.67:38187 (local address 127.0.0.1:49992)
0619 02:42:32.532316 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:32.532326 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:32.532379 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:32.532859 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:32.532860 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:32.532863 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:42:32.532900 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:32.532902 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:32.533805 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:32.533806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:32.534971 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:32.534974 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:32.535863 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:32.535864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:32.535930 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:32.535933 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:42:32.535982 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 02:42:32.536040 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"mutex_wait_us":29,"thread_start_us":53,"threads_started":1}
I20260619 02:42:32.536326 21932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:32.531948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49992 (local address 127.5.241.67:38187)
0619 02:42:32.532500 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:32.532504 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:32.532514 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:42:32.532552 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:32.532555 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:32.532571 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:32.532853 (+ 282us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:42:32.532969 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:32.533716 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:32.535052 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:32.535707 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:32.535776 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:32.535830 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:32.536180 (+ 350us) server_negotiation.cc:299] Negotiation successful
0619 02:42:32.536239 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":94,"threads_started":1}
I20260619 02:42:32.536650 21933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:32.531999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.66:36607 (local address 127.0.0.1:47814)
0619 02:42:32.532702 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:32.532711 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:32.532760 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:32.533019 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:32.533020 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:32.533022 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:42:32.533069 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:32.533071 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:32.534324 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:32.534325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:32.535405 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:32.535407 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:32.536385 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:32.536386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:32.536496 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:32.536501 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 02:42:32.536551 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 02:42:32.536590 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":692,"thread_start_us":63,"threads_started":1}
I20260619 02:42:32.536944 21934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:32.532141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47814 (local address 127.5.241.66:36607)
0619 02:42:32.532289 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:32.532294 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:32.532798 (+ 504us) server_negotiation.cc:407] Connection header received
0619 02:42:32.532847 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:32.532850 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:32.532866 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:32.532937 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:42:32.533148 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:32.534221 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:32.535473 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:32.536235 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:32.536301 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:32.536350 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:32.536823 (+ 473us) server_negotiation.cc:299] Negotiation successful
0619 02:42:32.536866 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":87,"threads_started":1}
I20260619 02:42:32.537017 21855 tablet_service.cc:1467] Processing CreateTablet for tablet e10fe4b59e8e4e3fbe329d11fda4b59c (DEFAULT_TABLE table=test-table [id=c7d65994707f41b095d3cc00e282ed46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:42:32.537186 21587 tablet_service.cc:1467] Processing CreateTablet for tablet e10fe4b59e8e4e3fbe329d11fda4b59c (DEFAULT_TABLE table=test-table [id=c7d65994707f41b095d3cc00e282ed46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:42:32.537381 21855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e10fe4b59e8e4e3fbe329d11fda4b59c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:42:32.537495 21587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e10fe4b59e8e4e3fbe329d11fda4b59c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:42:32.537554 21721 tablet_service.cc:1467] Processing CreateTablet for tablet e10fe4b59e8e4e3fbe329d11fda4b59c (DEFAULT_TABLE table=test-table [id=c7d65994707f41b095d3cc00e282ed46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:42:32.537822 21721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e10fe4b59e8e4e3fbe329d11fda4b59c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:42:32.539997 21935 tablet_bootstrap.cc:492] T e10fe4b59e8e4e3fbe329d11fda4b59c P 51b5ade6b85d4ab2976d15aea366ef09: Bootstrap starting.
I20260619 02:42:32.540663 21935 tablet_bootstrap.cc:654] T e10fe4b59e8e4e3fbe329d11fda4b59c P 51b5ade6b85d4ab2976d15aea366ef09: Neither blocks nor log segments found. Creating new log.
I20260619 02:42:32.540783 21936 tablet_bootstrap.cc:492] T e10fe4b59e8e4e3fbe329d11fda4b59c P 1c084879cb83420590952a75fa02ff81: Bootstrap starting.
I20260619 02:42:32.541085 21937 tablet_bootstrap.cc:492] T e10fe4b59e8e4e3fbe329d11fda4b59c P 1e6244cc32344d32ba646eb8c9a6f965: Bootstrap starting.
I20260619 02:42:32.541242 21935 log.cc:824] T e10fe4b59e8e4e3fbe329d11fda4b59c P 51b5ade6b85d4ab2976d15aea366ef09: Log is configured to *not* fsync() on all Append() calls
I20260619 02:42:32.541442 21936 tablet_bootstrap.cc:654] T e10fe4b59e8e4e3fbe329d11fda4b59c P 1c084879cb83420590952a75fa02ff81: Neither blocks nor log segments found. Creating new log.
I20260619 02:42:32.541723 21937 tablet_bootstrap.cc:654] T e10fe4b59e8e4e3fbe329d11fda4b59c P 1e6244cc32344d32ba646eb8c9a6f965: Neither blocks nor log segments found. Creating new log.
I20260619 02:42:32.541994 21936 log.cc:824] T e10fe4b59e8e4e3fbe329d11fda4b59c P 1c084879cb83420590952a75fa02ff81: Log is configured to *not* fsync() on all Append() calls
I20260619 02:42:32.542158 21935 tablet_bootstrap.cc:492] T e10fe4b59e8e4e3fbe329d11fda4b59c P 51b5ade6b85d4ab2976d15aea366ef09: No bootstrap required, opened a new log
I20260619 02:42:32.542233 21935 ts_tablet_manager.cc:1397] T e10fe4b59e8e4e3fbe329d11fda4b59c P 51b5ade6b85d4ab2976d15aea366ef09: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:42:32.542277 21937 log.cc:824] T e10fe4b59e8e4e3fbe329d11fda4b59c P 1e6244cc32344d32ba646eb8c9a6f965: Log is configured to *not* fsync() on all Append() calls
I20260619 02:42:32.542899 21936 tablet_bootstrap.cc:492] T e10fe4b59e8e4e3fbe329d11fda4b59c P 1c084879cb83420590952a75fa02ff81: No bootstrap required, opened a new log
I20260619 02:42:32.543015 21937 tablet_bootstrap.cc:492] T e10fe4b59e8e4e3fbe329d11fda4b59c P 1e6244cc32344d32ba646eb8c9a6f965: No bootstrap required, opened a new log
I20260619 02:42:32.543078 21937 ts_tablet_manager.cc:1397] T e10fe4b59e8e4e3fbe329d11fda4b59c P 1e6244cc32344d32ba646eb8c9a6f965: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:42:32.543170 21936 ts_tablet_manager.cc:1397] T e10fe4b59e8e4e3fbe329d11fda4b59c P 1c084879cb83420590952a75fa02ff81: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:42:32.544359 21935 raft_consensus.cc:348] T e10fe4b59e8e4e3fbe329d11fda4b59c P 51b5ade6b85d4ab2976d15aea366ef09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c084879cb83420590952a75fa02ff81" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 39547 } } peers { permanent_uuid: "51b5ade6b85d4ab2976d15aea366ef09" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38187 } } peers { permanent_uuid: "1e6244cc32344d32ba646eb8c9a6f965" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36607 } }
I20260619 02:42:32.544539 21935 raft_consensus.cc:374] T e10fe4b59e8e4e3fbe329d11fda4b59c P 51b5ade6b85d4ab2976d15aea366ef09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:42:32.544586 21935 raft_consensus.cc:729] T e10fe4b59e8e4e3fbe329d11fda4b59c P 51b5ade6b85d4ab2976d15aea366ef09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51b5ade6b85d4ab2976d15aea366ef09, State: Initialized, Role: FOLLOWER
I20260619 02:42:32.544605 21936 raft_consensus.cc:348] T e10fe4b59e8e4e3fbe329d11fda4b59c P 1c084879cb83420590952a75fa02ff81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c084879cb83420590952a75fa02ff81" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 39547 } } peers { permanent_uuid: "51b5ade6b85d4ab2976d15aea366ef09" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38187 } } peers { permanent_uuid: "1e6244cc32344d32ba646eb8c9a6f965" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36607 } }
I20260619 02:42:32.544723 21935 consensus_queue.cc:260] T e10fe4b59e8e4e3fbe329d11fda4b59c P 51b5ade6b85d4ab2976d15aea366ef09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c084879cb83420590952a75fa02ff81" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 39547 } } peers { permanent_uuid: "51b5ade6b85d4ab2976d15aea366ef09" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38187 } } peers { permanent_uuid: "1e6244cc32344d32ba646eb8c9a6f965" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36607 } }
I20260619 02:42:32.544787 21936 raft_consensus.cc:374] T e10fe4b59e8e4e3fbe329d11fda4b59c P 1c084879cb83420590952a75fa02ff81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:42:32.544833 21936 raft_consensus.cc:729] T e10fe4b59e8e4e3fbe329d11fda4b59c P 1c084879cb83420590952a75fa02ff81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c084879cb83420590952a75fa02ff81, State: Initialized, Role: FOLLOWER
I20260619 02:42:32.544952 21936 consensus_queue.cc:260] T e10fe4b59e8e4e3fbe329d11fda4b59c P 1c084879cb83420590952a75fa02ff81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c084879cb83420590952a75fa02ff81" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 39547 } } peers { permanent_uuid: "51b5ade6b85d4ab2976d15aea366ef09" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38187 } } peers { permanent_uuid: "1e6244cc32344d32ba646eb8c9a6f965" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36607 } }
I20260619 02:42:32.544989 21935 ts_tablet_manager.cc:1428] T e10fe4b59e8e4e3fbe329d11fda4b59c P 51b5ade6b85d4ab2976d15aea366ef09: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 02:42:32.544994 21937 raft_consensus.cc:348] T e10fe4b59e8e4e3fbe329d11fda4b59c P 1e6244cc32344d32ba646eb8c9a6f965 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c084879cb83420590952a75fa02ff81" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 39547 } } peers { permanent_uuid: "51b5ade6b85d4ab2976d15aea366ef09" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38187 } } peers { permanent_uuid: "1e6244cc32344d32ba646eb8c9a6f965" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36607 } }
I20260619 02:42:32.545138 21921 heartbeater.cc:503] Master 127.5.241.126:39127 was elected leader, sending a full tablet report...
I20260619 02:42:32.545146 21937 raft_consensus.cc:374] T e10fe4b59e8e4e3fbe329d11fda4b59c P 1e6244cc32344d32ba646eb8c9a6f965 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:42:32.545181 21937 raft_consensus.cc:729] T e10fe4b59e8e4e3fbe329d11fda4b59c P 1e6244cc32344d32ba646eb8c9a6f965 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e6244cc32344d32ba646eb8c9a6f965, State: Initialized, Role: FOLLOWER
I20260619 02:42:32.545225 21936 ts_tablet_manager.cc:1428] T e10fe4b59e8e4e3fbe329d11fda4b59c P 1c084879cb83420590952a75fa02ff81: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:42:32.545323 21653 heartbeater.cc:503] Master 127.5.241.126:39127 was elected leader, sending a full tablet report...
I20260619 02:42:32.545292 21937 consensus_queue.cc:260] T e10fe4b59e8e4e3fbe329d11fda4b59c P 1e6244cc32344d32ba646eb8c9a6f965 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c084879cb83420590952a75fa02ff81" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 39547 } } peers { permanent_uuid: "51b5ade6b85d4ab2976d15aea366ef09" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38187 } } peers { permanent_uuid: "1e6244cc32344d32ba646eb8c9a6f965" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36607 } }
I20260619 02:42:32.545540 21937 ts_tablet_manager.cc:1428] T e10fe4b59e8e4e3fbe329d11fda4b59c P 1e6244cc32344d32ba646eb8c9a6f965: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 02:42:32.545674 21787 heartbeater.cc:503] Master 127.5.241.126:39127 was elected leader, sending a full tablet report...
I20260619 02:42:32.600147 21941 raft_consensus.cc:482] T e10fe4b59e8e4e3fbe329d11fda4b59c P 51b5ade6b85d4ab2976d15aea366ef09 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:42:32.600278 21941 raft_consensus.cc:504] T e10fe4b59e8e4e3fbe329d11fda4b59c P 51b5ade6b85d4ab2976d15aea366ef09 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c084879cb83420590952a75fa02ff81" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 39547 } } peers { permanent_uuid: "51b5ade6b85d4ab2976d15aea366ef09" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38187 } } peers { permanent_uuid: "1e6244cc32344d32ba646eb8c9a6f965" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36607 } }
I20260619 02:42:32.600689 21941 leader_election.cc:288] T e10fe4b59e8e4e3fbe329d11fda4b59c P 51b5ade6b85d4ab2976d15aea366ef09 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c084879cb83420590952a75fa02ff81 (127.5.241.65:39547), 1e6244cc32344d32ba646eb8c9a6f965 (127.5.241.66:36607)
W20260619 02:42:32.602712 21788 tablet.cc:2369] T e10fe4b59e8e4e3fbe329d11fda4b59c P 1e6244cc32344d32ba646eb8c9a6f965: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:42:32.603513 21923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:32.600850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:39547 (local address 127.5.241.67:58919)
0619 02:42:32.600928 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:32.600942 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:32.600999 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:32.601171 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:32.601172 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:32.601173 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:42:32.601248 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:32.601249 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:32.601900 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:32.601901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:32.602645 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:32.602647 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:32.603257 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:32.603257 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:32.603337 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:32.603340 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:42:32.603397 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 02:42:32.603441 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260619 02:42:32.603513 21931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:32.601001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:58919 (local address 127.5.241.65:39547)
0619 02:42:32.601047 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:32.601049 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:32.601057 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:42:32.601094 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:32.601094 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:32.601104 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:32.601166 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:42:32.601324 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:32.601817 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:32.602736 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:32.603175 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:32.603259 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:32.603268 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:32.603416 (+ 148us) server_negotiation.cc:299] Negotiation successful
0619 02:42:32.603452 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 02:42:32.603837 21934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:32.601029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:51025 (local address 127.5.241.66:36607)
0619 02:42:32.601073 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:32.601075 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:32.601210 (+ 135us) server_negotiation.cc:407] Connection header received
0619 02:42:32.601282 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:32.601283 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:32.601292 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:32.601358 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:42:32.601509 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:32.601985 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:32.603101 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:32.603490 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:32.603571 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:32.603579 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:32.603740 (+ 161us) server_negotiation.cc:299] Negotiation successful
0619 02:42:32.603776 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:42:32.603885 21607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e10fe4b59e8e4e3fbe329d11fda4b59c" candidate_uuid: "51b5ade6b85d4ab2976d15aea366ef09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c084879cb83420590952a75fa02ff81" is_pre_election: true
I20260619 02:42:32.604055 21607 raft_consensus.cc:2454] T e10fe4b59e8e4e3fbe329d11fda4b59c P 1c084879cb83420590952a75fa02ff81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51b5ade6b85d4ab2976d15aea366ef09 in term 0.
I20260619 02:42:32.604070 21944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:32.600954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.66:36607 (local address 127.5.241.67:51025)
0619 02:42:32.601127 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:32.601143 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:32.601226 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:32.601360 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:32.601361 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:32.601362 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:42:32.601443 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:32.601445 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:32.602063 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:32.602064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:32.602997 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:32.603000 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:32.603573 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:32.603574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:32.603653 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:32.603657 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:42:32.603937 (+ 280us) client_negotiation.cc:240] Negotiation successful
0619 02:42:32.603983 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":111,"threads_started":1}
I20260619 02:42:32.604279 21810 leader_election.cc:302] T e10fe4b59e8e4e3fbe329d11fda4b59c P 51b5ade6b85d4ab2976d15aea366ef09 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c084879cb83420590952a75fa02ff81, 51b5ade6b85d4ab2976d15aea366ef09; no voters:
I20260619 02:42:32.604400 21941 raft_consensus.cc:2781] T e10fe4b59e8e4e3fbe329d11fda4b59c P 51b5ade6b85d4ab2976d15aea366ef09 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:42:32.604404 21741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e10fe4b59e8e4e3fbe329d11fda4b59c" candidate_uuid: "51b5ade6b85d4ab2976d15aea366ef09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e6244cc32344d32ba646eb8c9a6f965" is_pre_election: true
I20260619 02:42:32.604454 21941 raft_consensus.cc:482] T e10fe4b59e8e4e3fbe329d11fda4b59c P 51b5ade6b85d4ab2976d15aea366ef09 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:42:32.604496 21941 raft_consensus.cc:3037] T e10fe4b59e8e4e3fbe329d11fda4b59c P 51b5ade6b85d4ab2976d15aea366ef09 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:42:32.604544 21741 raft_consensus.cc:2454] T e10fe4b59e8e4e3fbe329d11fda4b59c P 1e6244cc32344d32ba646eb8c9a6f965 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51b5ade6b85d4ab2976d15aea366ef09 in term 0.
I20260619 02:42:32.605243 21941 raft_consensus.cc:504] T e10fe4b59e8e4e3fbe329d11fda4b59c P 51b5ade6b85d4ab2976d15aea366ef09 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c084879cb83420590952a75fa02ff81" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 39547 } } peers { permanent_uuid: "51b5ade6b85d4ab2976d15aea366ef09" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38187 } } peers { permanent_uuid: "1e6244cc32344d32ba646eb8c9a6f965" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36607 } }
I20260619 02:42:32.605365 21941 leader_election.cc:288] T e10fe4b59e8e4e3fbe329d11fda4b59c P 51b5ade6b85d4ab2976d15aea366ef09 [CANDIDATE]: Term 1 election: Requested vote from peers 1c084879cb83420590952a75fa02ff81 (127.5.241.65:39547), 1e6244cc32344d32ba646eb8c9a6f965 (127.5.241.66:36607)
I20260619 02:42:32.605526 21607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e10fe4b59e8e4e3fbe329d11fda4b59c" candidate_uuid: "51b5ade6b85d4ab2976d15aea366ef09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c084879cb83420590952a75fa02ff81"
I20260619 02:42:32.605572 21741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e10fe4b59e8e4e3fbe329d11fda4b59c" candidate_uuid: "51b5ade6b85d4ab2976d15aea366ef09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e6244cc32344d32ba646eb8c9a6f965"
I20260619 02:42:32.605618 21607 raft_consensus.cc:3037] T e10fe4b59e8e4e3fbe329d11fda4b59c P 1c084879cb83420590952a75fa02ff81 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:42:32.605667 21741 raft_consensus.cc:3037] T e10fe4b59e8e4e3fbe329d11fda4b59c P 1e6244cc32344d32ba646eb8c9a6f965 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:42:32.606307 21607 raft_consensus.cc:2454] T e10fe4b59e8e4e3fbe329d11fda4b59c P 1c084879cb83420590952a75fa02ff81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51b5ade6b85d4ab2976d15aea366ef09 in term 1.
I20260619 02:42:32.606354 21741 raft_consensus.cc:2454] T e10fe4b59e8e4e3fbe329d11fda4b59c P 1e6244cc32344d32ba646eb8c9a6f965 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51b5ade6b85d4ab2976d15aea366ef09 in term 1.
I20260619 02:42:32.606488 21810 leader_election.cc:302] T e10fe4b59e8e4e3fbe329d11fda4b59c P 51b5ade6b85d4ab2976d15aea366ef09 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c084879cb83420590952a75fa02ff81, 51b5ade6b85d4ab2976d15aea366ef09; no voters:
I20260619 02:42:32.606578 21941 raft_consensus.cc:2781] T e10fe4b59e8e4e3fbe329d11fda4b59c P 51b5ade6b85d4ab2976d15aea366ef09 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:42:32.606722 21941 raft_consensus.cc:686] T e10fe4b59e8e4e3fbe329d11fda4b59c P 51b5ade6b85d4ab2976d15aea366ef09 [term 1 LEADER]: Becoming Leader. State: Replica: 51b5ade6b85d4ab2976d15aea366ef09, State: Running, Role: LEADER
I20260619 02:42:32.606830 21941 consensus_queue.cc:237] T e10fe4b59e8e4e3fbe329d11fda4b59c P 51b5ade6b85d4ab2976d15aea366ef09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c084879cb83420590952a75fa02ff81" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 39547 } } peers { permanent_uuid: "51b5ade6b85d4ab2976d15aea366ef09" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38187 } } peers { permanent_uuid: "1e6244cc32344d32ba646eb8c9a6f965" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36607 } }
I20260619 02:42:32.607743 21462 catalog_manager.cc:5495] T e10fe4b59e8e4e3fbe329d11fda4b59c P 51b5ade6b85d4ab2976d15aea366ef09 reported cstate change: term changed from 0 to 1, leader changed from <none> to 51b5ade6b85d4ab2976d15aea366ef09 (127.5.241.67). New cstate: current_term: 1 leader_uuid: "51b5ade6b85d4ab2976d15aea366ef09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c084879cb83420590952a75fa02ff81" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 39547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51b5ade6b85d4ab2976d15aea366ef09" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38187 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1e6244cc32344d32ba646eb8c9a6f965" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36607 } health_report { overall_health: UNKNOWN } } }
I20260619 02:42:32.629917 21932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:32.627787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50008 (local address 127.5.241.67:38187)
0619 02:42:32.627830 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:32.627832 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:32.627839 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:42:32.627909 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:32.627909 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:32.627925 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:32.627983 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:42:32.628095 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:32.628552 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:32.629249 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:32.629431 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:32.629479 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:32.629600 (+ 121us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:42:32.629691 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:42:32.629747 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:32.629777 (+ 30us) server_negotiation.cc:299] Negotiation successful
0619 02:42:32.629793 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 02:42:32.631892 21607 raft_consensus.cc:1261] T e10fe4b59e8e4e3fbe329d11fda4b59c P 1c084879cb83420590952a75fa02ff81 [term 1 FOLLOWER]: Refusing update from remote peer 51b5ade6b85d4ab2976d15aea366ef09: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:42:32.631892 21741 raft_consensus.cc:1261] T e10fe4b59e8e4e3fbe329d11fda4b59c P 1e6244cc32344d32ba646eb8c9a6f965 [term 1 FOLLOWER]: Refusing update from remote peer 51b5ade6b85d4ab2976d15aea366ef09: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:42:32.632298 21945 consensus_queue.cc:1035] T e10fe4b59e8e4e3fbe329d11fda4b59c P 51b5ade6b85d4ab2976d15aea366ef09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c084879cb83420590952a75fa02ff81" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 39547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:42:32.632441 21941 consensus_queue.cc:1035] T e10fe4b59e8e4e3fbe329d11fda4b59c P 51b5ade6b85d4ab2976d15aea366ef09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e6244cc32344d32ba646eb8c9a6f965" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:42:32.635042 21950 mvcc.cc:205] Tried to move back new op lower bound from 7298404157977059328 to 7298404157878292480. Current Snapshot: MvccSnapshot[applied={T|T < 7298404157977059328}]
I20260619 02:42:32.635845 21949 mvcc.cc:205] Tried to move back new op lower bound from 7298404157977059328 to 7298404157878292480. Current Snapshot: MvccSnapshot[applied={T|T < 7298404157977059328}]
I20260619 02:42:32.637111 21462 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43074:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:42:32.637212 21462 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43074:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:42:32.638170 21462 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190: Sending DeleteTablet for 3 replicas of tablet e10fe4b59e8e4e3fbe329d11fda4b59c
I20260619 02:42:32.638525 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 21524
I20260619 02:42:32.638509 21587 tablet_service.cc:1514] Processing DeleteTablet for tablet e10fe4b59e8e4e3fbe329d11fda4b59c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:42:32 UTC) from {username='kudu', principal='kudu/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:48852
I20260619 02:42:32.638533 21855 tablet_service.cc:1514] Processing DeleteTablet for tablet e10fe4b59e8e4e3fbe329d11fda4b59c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:42:32 UTC) from {username='kudu', principal='kudu/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:49992
I20260619 02:42:32.638574 21721 tablet_service.cc:1514] Processing DeleteTablet for tablet e10fe4b59e8e4e3fbe329d11fda4b59c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:42:32 UTC) from {username='kudu', principal='kudu/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:47814
I20260619 02:42:32.640638 21959 tablet_replica.cc:330] T e10fe4b59e8e4e3fbe329d11fda4b59c P 1e6244cc32344d32ba646eb8c9a6f965: stopping tablet replica
I20260619 02:42:32.640753 21959 raft_consensus.cc:2229] T e10fe4b59e8e4e3fbe329d11fda4b59c P 1e6244cc32344d32ba646eb8c9a6f965 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:42:32.640868 21959 raft_consensus.cc:2258] T e10fe4b59e8e4e3fbe329d11fda4b59c P 1e6244cc32344d32ba646eb8c9a6f965 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:42:32.641023 21958 tablet_replica.cc:330] T e10fe4b59e8e4e3fbe329d11fda4b59c P 51b5ade6b85d4ab2976d15aea366ef09: stopping tablet replica
I20260619 02:42:32.641117 21958 raft_consensus.cc:2229] T e10fe4b59e8e4e3fbe329d11fda4b59c P 51b5ade6b85d4ab2976d15aea366ef09 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:42:32.641206 21958 raft_consensus.cc:2258] T e10fe4b59e8e4e3fbe329d11fda4b59c P 51b5ade6b85d4ab2976d15aea366ef09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:42:32.641593 21959 ts_tablet_manager.cc:1905] T e10fe4b59e8e4e3fbe329d11fda4b59c P 1e6244cc32344d32ba646eb8c9a6f965: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:42:32.643569 21959 ts_tablet_manager.cc:1918] T e10fe4b59e8e4e3fbe329d11fda4b59c P 1e6244cc32344d32ba646eb8c9a6f965: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:42:32.643648 21959 log.cc:1196] T e10fe4b59e8e4e3fbe329d11fda4b59c P 1e6244cc32344d32ba646eb8c9a6f965: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-1/wal/wals/e10fe4b59e8e4e3fbe329d11fda4b59c
I20260619 02:42:32.643952 21959 ts_tablet_manager.cc:1939] T e10fe4b59e8e4e3fbe329d11fda4b59c P 1e6244cc32344d32ba646eb8c9a6f965: Deleting consensus metadata
W20260619 02:42:32.643949 21448 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.241.65:39547 (error 108)
I20260619 02:42:32.644347 21448 catalog_manager.cc:4841] TS 1e6244cc32344d32ba646eb8c9a6f965 (127.5.241.66:36607): tablet e10fe4b59e8e4e3fbe329d11fda4b59c (table test-table [id=c7d65994707f41b095d3cc00e282ed46]) successfully deleted
I20260619 02:42:32.644488 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 21658
I20260619 02:42:32.644526 21958 ts_tablet_manager.cc:1905] T e10fe4b59e8e4e3fbe329d11fda4b59c P 51b5ade6b85d4ab2976d15aea366ef09: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:42:32.646413 21958 ts_tablet_manager.cc:1918] T e10fe4b59e8e4e3fbe329d11fda4b59c P 51b5ade6b85d4ab2976d15aea366ef09: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:42:32.646490 21958 log.cc:1196] T e10fe4b59e8e4e3fbe329d11fda4b59c P 51b5ade6b85d4ab2976d15aea366ef09: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-2/wal/wals/e10fe4b59e8e4e3fbe329d11fda4b59c
I20260619 02:42:32.646759 21958 ts_tablet_manager.cc:1939] T e10fe4b59e8e4e3fbe329d11fda4b59c P 51b5ade6b85d4ab2976d15aea366ef09: Deleting consensus metadata
I20260619 02:42:32.647188 21447 catalog_manager.cc:4841] TS 51b5ade6b85d4ab2976d15aea366ef09 (127.5.241.67:38187): tablet e10fe4b59e8e4e3fbe329d11fda4b59c (table test-table [id=c7d65994707f41b095d3cc00e282ed46]) successfully deleted
I20260619 02:42:32.650220 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 21792
I20260619 02:42:32.650262 21933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:32.650127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:39547 (local address 127.0.0.1:48860)
0619 02:42:32.650190 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:32.650227 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.65:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260619 02:42:32.650926 21448 catalog_manager.cc:4584] TS 1c084879cb83420590952a75fa02ff81 (127.5.241.65:39547): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e10fe4b59e8e4e3fbe329d11fda4b59c: Network error: Client connection negotiation failed: client connection to 127.5.241.65:39547: connect: Connection refused (error 111)
I20260619 02:42:32.654889 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 21430
I20260619 02:42:52.661794 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:39127
--webserver_interface=127.5.241.126
--webserver_port=34563
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:44479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:39127
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:42:52.743283 21961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:42:52.743518 21961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:42:52.743542 21961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:42:52.744738 21961 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 02:42:52.744782 21961 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 02:42:52.744796 21961 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 02:42:52.744808 21961 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:42:52.744822 21961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:42:52.744879 21961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:42:52.744896 21961 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:42:52.744920 21961 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:42:52.746418 21961 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:44479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:39127
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:39127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.241.126
--webserver_port=34563
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21961
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:42:52.746665 21961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:42:52.746927 21961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:42:52.755885 21968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:52.755908 21967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:42:52.755906 21961 server_base.cc:1034] running on GCE node
W20260619 02:42:52.755884 21970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:42:52.756332 21961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:42:52.756628 21961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:42:52.757800 21961 hybrid_clock.cc:648] HybridClock initialized: now 1781836972757790 us; error 33 us; skew 500 ppm
Jun 19 02:42:52 dist-test-slave-zpbg krb5kdc[21417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836972, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:42:52.767520 21961 init.cc:377] Logged in from keytab as kudu/127.5.241.126@KRBTEST.COM (short username kudu)
I20260619 02:42:52.768064 21961 webserver.cc:460] Webserver started at http://127.5.241.126:34563/ using document root <none> and password file <none>
I20260619 02:42:52.768309 21961 fs_manager.cc:359] Metadata directory not provided
I20260619 02:42:52.768401 21961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:42:52.769829 21961 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:42:52.770646 21977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:52.770915 21961 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:42:52.770998 21961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "6c315d868f78434a8e1c444c0279e190"
format_stamp: "Formatted at 2026-06-19 02:42:31 on dist-test-slave-zpbg"
server_key: "db9e53fe736cf602ca04bf799dbfb167"
server_key_iv: "135d364f0237f5781bdfbfb89800aee9"
server_key_version: "encryptionkey@0"
I20260619 02:42:52.771396 21961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:42:52.783372 21961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:42:52.785646 21961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:42:52.785755 21961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:42:52.789767 21961 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.126:39127
I20260619 02:42:52.789876 22029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.126:39127 every 8 connection(s)
I20260619 02:42:52.790239 21961 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:42:52.791918 22030 sys_catalog.cc:263] Verifying existing consensus state
I20260619 02:42:52.792613 22030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190: Bootstrap starting.
I20260619 02:42:52.797304 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 21961
I20260619 02:42:52.797632 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.65:39547
--local_ip_for_outbound_sockets=127.5.241.65
--tserver_master_addrs=127.5.241.126:39127
--webserver_port=35705
--webserver_interface=127.5.241.65
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab
--principal=kudu/127.5.241.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:44479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:42:52.802189 22030 log.cc:824] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190: Log is configured to *not* fsync() on all Append() calls
I20260619 02:42:52.804239 22030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:42:52.804556 22030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190: Bootstrap complete.
I20260619 02:42:52.806890 22030 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c315d868f78434a8e1c444c0279e190" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 39127 } }
I20260619 02:42:52.807160 22030 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c315d868f78434a8e1c444c0279e190, State: Initialized, Role: FOLLOWER
I20260619 02:42:52.807350 22030 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c315d868f78434a8e1c444c0279e190" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 39127 } }
I20260619 02:42:52.807432 22030 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:42:52.807492 22030 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:42:52.807540 22030 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190 [term 1 FOLLOWER]: Advancing to term 2
I20260619 02:42:52.808537 22030 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c315d868f78434a8e1c444c0279e190" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 39127 } }
I20260619 02:42:52.808662 22030 leader_election.cc:302] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c315d868f78434a8e1c444c0279e190; no voters:
I20260619 02:42:52.808871 22030 leader_election.cc:288] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 02:42:52.808912 22034 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 02:42:52.809028 22034 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190 [term 2 LEADER]: Becoming Leader. State: Replica: 6c315d868f78434a8e1c444c0279e190, State: Running, Role: LEADER
I20260619 02:42:52.809170 22030 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:42:52.809175 22034 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c315d868f78434a8e1c444c0279e190" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 39127 } }
I20260619 02:42:52.809576 22035 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "6c315d868f78434a8e1c444c0279e190" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c315d868f78434a8e1c444c0279e190" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 39127 } } }
I20260619 02:42:52.809649 22035 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190 [sys.catalog]: This master's current role is: LEADER
I20260619 02:42:52.810048 22036 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c315d868f78434a8e1c444c0279e190. Latest consensus state: current_term: 2 leader_uuid: "6c315d868f78434a8e1c444c0279e190" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c315d868f78434a8e1c444c0279e190" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 39127 } } }
I20260619 02:42:52.810127 22036 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190 [sys.catalog]: This master's current role is: LEADER
I20260619 02:42:52.810273 22040 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:42:52.811017 22040 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:42:52.811272 22040 catalog_manager.cc:1259] Loaded cluster ID: 21a3f16bf7204b8e8b7a4af68209b14f
I20260619 02:42:52.811367 22040 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:42:52.812589 22040 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:42:52.812765 22040 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190: Loaded TSK: 0
I20260619 02:42:52.825709 22040 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190: Generated new TSK 1
I20260619 02:42:52.825826 22040 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 02:42:52.885425 22032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:42:52.885627 22032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:42:52.885651 22032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:42:52.886858 22032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:42:52.886900 22032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:42:52.886927 22032 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.65
I20260619 02:42:52.888370 22032 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:44479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab
--principal=kudu/127.5.241.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.65:39547
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.241.65
--webserver_port=35705
--tserver_master_addrs=127.5.241.126:39127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22032
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.65
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:42:52.888608 22032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:42:52.888866 22032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:42:52.893903 22061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:52.893893 22058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:52.894069 22059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:42:52.894285 22032 server_base.cc:1034] running on GCE node
I20260619 02:42:52.894505 22032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:42:52.894757 22032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:42:52.895946 22032 hybrid_clock.cc:648] HybridClock initialized: now 1781836972895935 us; error 32 us; skew 500 ppm
Jun 19 02:42:52 dist-test-slave-zpbg krb5kdc[21417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836972, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:42:52.898546 22032 init.cc:377] Logged in from keytab as kudu/127.5.241.65@KRBTEST.COM (short username kudu)
I20260619 02:42:52.899075 22032 webserver.cc:460] Webserver started at http://127.5.241.65:35705/ using document root <none> and password file <none>
I20260619 02:42:52.899335 22032 fs_manager.cc:359] Metadata directory not provided
I20260619 02:42:52.899392 22032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:42:52.900763 22032 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260619 02:42:52.901458 22068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:52.901608 22032 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:42:52.901711 22032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-0/wal
uuid: "1c084879cb83420590952a75fa02ff81"
format_stamp: "Formatted at 2026-06-19 02:42:32 on dist-test-slave-zpbg"
server_key: "de1ee857088991bde4e566a46099bd1f"
server_key_iv: "bed7176c6ba9e35235a3f6062a0957a7"
server_key_version: "encryptionkey@0"
I20260619 02:42:52.902053 22032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:42:52.912468 22032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:42:52.914944 22032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:42:52.915100 22032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:42:52.915392 22032 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:42:52.915977 22075 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 02:42:52.917557 22032 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 02:42:52.917618 22032 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 02:42:52.917656 22032 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 02:42:52.918540 22032 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 02:42:52.918586 22032 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 02:42:52.918694 22075 tablet_bootstrap.cc:492] T e10fe4b59e8e4e3fbe329d11fda4b59c P 1c084879cb83420590952a75fa02ff81: Bootstrap starting.
I20260619 02:42:52.926143 22032 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.65:39547
I20260619 02:42:52.926381 22182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.65:39547 every 8 connection(s)
I20260619 02:42:52.926615 22032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
I20260619 02:42:52.930269 22075 log.cc:824] T e10fe4b59e8e4e3fbe329d11fda4b59c P 1c084879cb83420590952a75fa02ff81: Log is configured to *not* fsync() on all Append() calls
I20260619 02:42:52.932768 22075 tablet_bootstrap.cc:492] T e10fe4b59e8e4e3fbe329d11fda4b59c P 1c084879cb83420590952a75fa02ff81: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:42:52.932806 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 22032
I20260619 02:42:52.933086 22075 tablet_bootstrap.cc:492] T e10fe4b59e8e4e3fbe329d11fda4b59c P 1c084879cb83420590952a75fa02ff81: Bootstrap complete.
I20260619 02:42:52.933102 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.66:36607
--local_ip_for_outbound_sockets=127.5.241.66
--tserver_master_addrs=127.5.241.126:39127
--webserver_port=44697
--webserver_interface=127.5.241.66
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab
--principal=kudu/127.5.241.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:44479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:42:52.934648 22075 ts_tablet_manager.cc:1397] T e10fe4b59e8e4e3fbe329d11fda4b59c P 1c084879cb83420590952a75fa02ff81: Time spent bootstrapping tablet: real 0.016s user 0.009s sys 0.006s
Jun 19 02:42:52 dist-test-slave-zpbg krb5kdc[21417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836972, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.65@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:42:52.935858 22075 raft_consensus.cc:348] T e10fe4b59e8e4e3fbe329d11fda4b59c P 1c084879cb83420590952a75fa02ff81 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c084879cb83420590952a75fa02ff81" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 39547 } } peers { permanent_uuid: "51b5ade6b85d4ab2976d15aea366ef09" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38187 } } peers { permanent_uuid: "1e6244cc32344d32ba646eb8c9a6f965" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36607 } }
I20260619 02:42:52.936115 22075 raft_consensus.cc:729] T e10fe4b59e8e4e3fbe329d11fda4b59c P 1c084879cb83420590952a75fa02ff81 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c084879cb83420590952a75fa02ff81, State: Initialized, Role: FOLLOWER
I20260619 02:42:52.936275 22075 consensus_queue.cc:260] T e10fe4b59e8e4e3fbe329d11fda4b59c P 1c084879cb83420590952a75fa02ff81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c084879cb83420590952a75fa02ff81" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 39547 } } peers { permanent_uuid: "51b5ade6b85d4ab2976d15aea366ef09" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38187 } } peers { permanent_uuid: "1e6244cc32344d32ba646eb8c9a6f965" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36607 } }
I20260619 02:42:52.936558 22075 ts_tablet_manager.cc:1428] T e10fe4b59e8e4e3fbe329d11fda4b59c P 1c084879cb83420590952a75fa02ff81: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 02:42:52.940582 22185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:52.927406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.65:56753)
0619 02:42:52.927617 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:52.927685 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:52.928387 (+ 702us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:52.928917 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:52.928921 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:52.929350 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:42:52.929683 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:52.929691 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:52.931389 (+ 1698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:52.931390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:52.932640 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:52.932643 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:52.933848 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:52.933849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:52.933999 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:52.934630 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:42:52.934640 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:42:52.936505 (+ 1865us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:42:52.938614 (+ 2109us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:52.938622 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:52.938625 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:52.938991 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:52.939301 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:52.939302 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:52.939302 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:52.939443 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:52.939929 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:42:52.939936 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:42:52.940085 (+ 149us) client_negotiation.cc:769] Sending connection context
0619 02:42:52.940198 (+ 113us) client_negotiation.cc:240] Negotiation successful
0619 02:42:52.940378 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":99,"threads_started":1}
I20260619 02:42:52.940688 22186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:52.927825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:56753 (local address 127.5.241.126:39127)
0619 02:42:52.928029 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:52.928038 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:52.928082 (+ 44us) server_negotiation.cc:407] Connection header received
0619 02:42:52.928633 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:52.928640 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:52.928700 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:52.928847 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:42:52.929821 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:52.931252 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:52.932740 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:52.933726 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:52.934291 (+ 565us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:52.936601 (+ 2310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:42:52.936606 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:42:52.936606 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:42:52.936617 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:42:52.938523 (+ 1906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:52.939090 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:52.939093 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:52.939093 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:52.939214 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:52.939522 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:52.939523 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:52.939524 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:52.939845 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:42:52.939932 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:52.940287 (+ 355us) server_negotiation.cc:299] Negotiation successful
0619 02:42:52.940446 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":92,"threads_started":1}
I20260619 02:42:52.941646 22183 heartbeater.cc:347] Connected to a master server at 127.5.241.126:39127
I20260619 02:42:52.941756 22183 heartbeater.cc:464] Registering TS with master...
I20260619 02:42:52.941964 22183 heartbeater.cc:511] Master 127.5.241.126:39127 requested a full tablet report, sending...
I20260619 02:42:52.942804 21994 ts_manager.cc:194] Registered new tserver with Master: 1c084879cb83420590952a75fa02ff81 (127.5.241.65:39547)
I20260619 02:42:52.943986 21994 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.65@KRBTEST.COM'} at 127.5.241.65:56753
I20260619 02:42:52.945034 22183 heartbeater.cc:503] Master 127.5.241.126:39127 was elected leader, sending a full tablet report...
I20260619 02:42:52.951179 22190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:52.943442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:39547 (local address 127.0.0.1:51622)
0619 02:42:52.946905 (+ 3463us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:52.946945 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:52.947029 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:52.947252 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:52.947255 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:52.947257 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:42:52.947327 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:52.947329 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:52.948757 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:52.948758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:52.949963 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:52.949967 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:52.950928 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:52.950929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:52.950992 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:52.951004 (+ 12us) client_negotiation.cc:769] Sending connection context
0619 02:42:52.951064 (+ 60us) client_negotiation.cc:240] Negotiation successful
0619 02:42:52.951109 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3429,"thread_start_us":107,"threads_started":1}
I20260619 02:42:52.951190 22191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:52.943716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51622 (local address 127.5.241.65:39547)
0619 02:42:52.946912 (+ 3196us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:52.946918 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:52.947026 (+ 108us) server_negotiation.cc:407] Connection header received
0619 02:42:52.947124 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:52.947127 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:52.947145 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:52.947237 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:42:52.947490 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:52.948657 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:52.950056 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:52.950848 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:52.950925 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:52.950982 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:52.951077 (+ 95us) server_negotiation.cc:299] Negotiation successful
0619 02:42:52.951115 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3156,"thread_start_us":75,"threads_started":1}
I20260619 02:42:52.951642 22117 tablet_service.cc:1514] Processing DeleteTablet for tablet e10fe4b59e8e4e3fbe329d11fda4b59c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:42:32 UTC) from {username='kudu', principal='kudu/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:51622
I20260619 02:42:52.952306 22192 tablet_replica.cc:330] stopping tablet replica
I20260619 02:42:52.952428 22192 raft_consensus.cc:2229] T e10fe4b59e8e4e3fbe329d11fda4b59c P 1c084879cb83420590952a75fa02ff81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:42:52.952521 22192 raft_consensus.cc:2258] T e10fe4b59e8e4e3fbe329d11fda4b59c P 1c084879cb83420590952a75fa02ff81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:42:52.952833 22192 ts_tablet_manager.cc:1905] T e10fe4b59e8e4e3fbe329d11fda4b59c P 1c084879cb83420590952a75fa02ff81: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:42:52.954896 22192 ts_tablet_manager.cc:1918] T e10fe4b59e8e4e3fbe329d11fda4b59c P 1c084879cb83420590952a75fa02ff81: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:42:52.954988 22192 log.cc:1196] T e10fe4b59e8e4e3fbe329d11fda4b59c P 1c084879cb83420590952a75fa02ff81: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-0/wal/wals/e10fe4b59e8e4e3fbe329d11fda4b59c
I20260619 02:42:52.955205 22192 ts_tablet_manager.cc:1939] T e10fe4b59e8e4e3fbe329d11fda4b59c P 1c084879cb83420590952a75fa02ff81: Deleting consensus metadata
I20260619 02:42:52.955938 21980 catalog_manager.cc:4841] TS 1c084879cb83420590952a75fa02ff81 (127.5.241.65:39547): tablet e10fe4b59e8e4e3fbe329d11fda4b59c (table test-table [id=c7d65994707f41b095d3cc00e282ed46]) successfully deleted
W20260619 02:42:53.021414 22188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:42:53.021695 22188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:42:53.021737 22188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:42:53.022938 22188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:42:53.023011 22188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:42:53.023057 22188 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.66
I20260619 02:42:53.024600 22188 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:44479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab
--principal=kudu/127.5.241.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.66:36607
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.241.66
--webserver_port=44697
--tserver_master_addrs=127.5.241.126:39127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22188
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.66
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:42:53.024873 22188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:42:53.025107 22188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:42:53.030130 22198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:42:53.030223 22188 server_base.cc:1034] running on GCE node
W20260619 02:42:53.030122 22197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:53.030123 22200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:42:53.030556 22188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:42:53.030807 22188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:42:53.031957 22188 hybrid_clock.cc:648] HybridClock initialized: now 1781836973031944 us; error 34 us; skew 500 ppm
Jun 19 02:42:53 dist-test-slave-zpbg krb5kdc[21417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836973, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:42:53.034370 22188 init.cc:377] Logged in from keytab as kudu/127.5.241.66@KRBTEST.COM (short username kudu)
I20260619 02:42:53.034888 22188 webserver.cc:460] Webserver started at http://127.5.241.66:44697/ using document root <none> and password file <none>
I20260619 02:42:53.035116 22188 fs_manager.cc:359] Metadata directory not provided
I20260619 02:42:53.035195 22188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:42:53.036532 22188 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:42:53.037192 22207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:53.037379 22188 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:42:53.037477 22188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-1/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-1/wal
uuid: "1e6244cc32344d32ba646eb8c9a6f965"
format_stamp: "Formatted at 2026-06-19 02:42:32 on dist-test-slave-zpbg"
server_key: "3d080433e7293677d7af8ae265138acf"
server_key_iv: "8ad93a6546cc48599734ef16cf090ab5"
server_key_version: "encryptionkey@0"
I20260619 02:42:53.037823 22188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:42:53.072026 22188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:42:53.074371 22188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:42:53.074533 22188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:42:53.074820 22188 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:42:53.075189 22188 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:42:53.075243 22188 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:53.075286 22188 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:42:53.075312 22188 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:53.080977 22188 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.66:36607
I20260619 02:42:53.081020 22320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.66:36607 every 8 connection(s)
I20260619 02:42:53.081347 22188 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
I20260619 02:42:53.082288 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 22188
I20260619 02:42:53.082599 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.67:38187
--local_ip_for_outbound_sockets=127.5.241.67
--tserver_master_addrs=127.5.241.126:39127
--webserver_port=35923
--webserver_interface=127.5.241.67
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab
--principal=kudu/127.5.241.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:44479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 02:42:53 dist-test-slave-zpbg krb5kdc[21417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836973, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.66@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:42:53.091154 22186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:53.082052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:56205 (local address 127.5.241.126:39127)
0619 02:42:53.082121 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:53.082123 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:53.082331 (+ 208us) server_negotiation.cc:407] Connection header received
0619 02:42:53.082987 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:53.082988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:53.083005 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:53.083085 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:42:53.083998 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:53.084506 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:53.085509 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:53.085730 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:53.085818 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:53.088247 (+ 2429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:42:53.088249 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:42:53.088249 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:42:53.088256 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:42:53.089720 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:53.090129 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:53.090130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:53.090130 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:53.090191 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:53.090450 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:53.090451 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:53.090451 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:53.090564 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:42:53.090634 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:53.090970 (+ 336us) server_negotiation.cc:299] Negotiation successful
0619 02:42:53.091061 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 02:42:53.091342 22323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:53.082050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.66:56205)
0619 02:42:53.082194 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:53.082235 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:53.082867 (+ 632us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:53.083138 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:53.083141 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:53.083695 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:42:53.083894 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:53.083899 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:53.084605 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:53.084606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:53.085392 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:53.085394 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:53.085851 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:53.085852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:53.086019 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:53.086520 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:42:53.086532 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:42:53.088133 (+ 1601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:42:53.089827 (+ 1694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:53.089833 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:53.089835 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:53.090052 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:53.090259 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:53.090260 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:53.090261 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:53.090372 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:53.090638 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:42:53.090644 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:42:53.090825 (+ 181us) client_negotiation.cc:769] Sending connection context
0619 02:42:53.090951 (+ 126us) client_negotiation.cc:240] Negotiation successful
0619 02:42:53.091108 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":74,"threads_started":1}
I20260619 02:42:53.091849 22321 heartbeater.cc:347] Connected to a master server at 127.5.241.126:39127
I20260619 02:42:53.091964 22321 heartbeater.cc:464] Registering TS with master...
I20260619 02:42:53.092136 22321 heartbeater.cc:511] Master 127.5.241.126:39127 requested a full tablet report, sending...
I20260619 02:42:53.092521 21994 ts_manager.cc:194] Registered new tserver with Master: 1e6244cc32344d32ba646eb8c9a6f965 (127.5.241.66:36607)
I20260619 02:42:53.092914 21994 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.66@KRBTEST.COM'} at 127.5.241.66:56205
W20260619 02:42:53.164458 22324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:42:53.164657 22324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:42:53.164681 22324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:42:53.165824 22324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:42:53.165866 22324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:42:53.165894 22324 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.67
I20260619 02:42:53.167481 22324 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:44479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab
--principal=kudu/127.5.241.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.67:38187
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.241.67
--webserver_port=35923
--tserver_master_addrs=127.5.241.126:39127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22324
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.67
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:42:53.167740 22324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:42:53.167987 22324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:42:53.173161 22329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:53.173260 22330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:53.173168 22332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:42:53.173470 22324 server_base.cc:1034] running on GCE node
I20260619 02:42:53.173684 22324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:42:53.173988 22324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:42:53.175153 22324 hybrid_clock.cc:648] HybridClock initialized: now 1781836973175150 us; error 39 us; skew 500 ppm
Jun 19 02:42:53 dist-test-slave-zpbg krb5kdc[21417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836973, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:42:53.177647 22324 init.cc:377] Logged in from keytab as kudu/127.5.241.67@KRBTEST.COM (short username kudu)
I20260619 02:42:53.178133 22324 webserver.cc:460] Webserver started at http://127.5.241.67:35923/ using document root <none> and password file <none>
I20260619 02:42:53.178362 22324 fs_manager.cc:359] Metadata directory not provided
I20260619 02:42:53.178447 22324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:42:53.179946 22324 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:42:53.180725 22339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:53.180958 22324 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:42:53.181067 22324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-2/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-2/wal
uuid: "51b5ade6b85d4ab2976d15aea366ef09"
format_stamp: "Formatted at 2026-06-19 02:42:32 on dist-test-slave-zpbg"
server_key: "78c5584faf61b792b41bd0209d810b76"
server_key_iv: "855742e05241d138923a434f6ef22ed2"
server_key_version: "encryptionkey@0"
I20260619 02:42:53.181466 22324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:42:53.192622 22324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:42:53.195290 22324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:42:53.195442 22324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:42:53.195708 22324 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:42:53.196020 22324 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:42:53.196074 22324 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:53.196117 22324 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:42:53.196144 22324 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:53.201861 22324 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.67:38187
I20260619 02:42:53.201961 22452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.67:38187 every 8 connection(s)
I20260619 02:42:53.202296 22324 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
I20260619 02:42:53.207690 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 22324
Jun 19 02:42:53 dist-test-slave-zpbg krb5kdc[21417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836973, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.67@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:42:53.212805 22186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:53.203080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:57533 (local address 127.5.241.126:39127)
0619 02:42:53.203123 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:53.203125 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:53.203318 (+ 193us) server_negotiation.cc:407] Connection header received
0619 02:42:53.203863 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:53.203865 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:53.203901 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:53.203961 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:42:53.204979 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:53.205518 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:53.206522 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:53.206723 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:53.206821 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:53.209263 (+ 2442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:42:53.209265 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:42:53.209265 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:42:53.209274 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:42:53.211176 (+ 1902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:53.211632 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:53.211633 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:53.211633 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:53.211695 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:53.212008 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:53.212009 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:53.212010 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:53.212165 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:42:53.212246 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:53.212665 (+ 419us) server_negotiation.cc:299] Negotiation successful
0619 02:42:53.212740 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 02:42:53.213058 22455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:53.203048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.67:57533)
0619 02:42:53.203187 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:53.203231 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:53.203748 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:53.204040 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:53.204044 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:53.204682 (+ 638us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:42:53.204877 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:53.204882 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:53.205620 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:53.205621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:53.206427 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:53.206429 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:53.206827 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:53.206829 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:53.206961 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:53.207527 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:42:53.207536 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:42:53.209154 (+ 1618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:42:53.211260 (+ 2106us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:53.211267 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:53.211269 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:53.211530 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:53.211763 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:53.211764 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:53.211764 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:53.211916 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:53.212267 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:42:53.212274 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:42:53.212474 (+ 200us) client_negotiation.cc:769] Sending connection context
0619 02:42:53.212612 (+ 138us) client_negotiation.cc:240] Negotiation successful
0619 02:42:53.212800 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":68,"threads_started":1}
I20260619 02:42:53.213536 22453 heartbeater.cc:347] Connected to a master server at 127.5.241.126:39127
I20260619 02:42:53.213629 22453 heartbeater.cc:464] Registering TS with master...
I20260619 02:42:53.213800 22453 heartbeater.cc:511] Master 127.5.241.126:39127 requested a full tablet report, sending...
I20260619 02:42:53.214236 21994 ts_manager.cc:194] Registered new tserver with Master: 51b5ade6b85d4ab2976d15aea366ef09 (127.5.241.67:38187)
I20260619 02:42:53.214597 21994 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.67@KRBTEST.COM'} at 127.5.241.67:57533
I20260619 02:42:53.216066 22457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:53.208327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45440 (local address 127.5.241.126:39127)
0619 02:42:53.208500 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:53.208502 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:53.208511 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:42:53.208578 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:53.208579 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:53.208590 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:53.208679 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:42:53.209536 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:53.210350 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:53.211329 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:53.211514 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:53.211565 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:53.213194 (+ 1629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:42:53.213195 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:42:53.213196 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:42:53.213204 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:42:53.214901 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:53.215283 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:53.215284 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:53.215285 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:53.215342 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:53.215552 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:53.215553 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:53.215554 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:53.215671 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:42:53.215851 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:53.215887 (+ 36us) server_negotiation.cc:299] Negotiation successful
0619 02:42:53.215967 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":64,"threads_started":1}
I20260619 02:42:53.216784 6085 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:42:53.220055 22457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:53.217205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45456 (local address 127.5.241.126:39127)
0619 02:42:53.217298 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:53.217300 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:53.217386 (+ 86us) server_negotiation.cc:407] Connection header received
0619 02:42:53.217470 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:53.217471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:53.217515 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:53.217598 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:42:53.217725 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:53.218182 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:53.219124 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:53.219396 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:53.219477 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:53.219660 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:42:53.219705 (+ 45us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0619 02:42:53.219990 (+ 285us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45456: token expired
Metrics: {"server-negotiator.queue_time_us":71}
W20260619 02:42:53.220105 22457 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45456: token expired
W20260619 02:42:53.220448 21926 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.5.241.126:39127)
I20260619 02:42:53.227097 22457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:53.220961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45464 (local address 127.5.241.126:39127)
0619 02:42:53.221001 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:53.221003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:53.221010 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:42:53.221029 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:53.221030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:53.221038 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:53.221091 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:42:53.221744 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:53.222256 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:53.223091 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:53.223297 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:53.223380 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:53.224563 (+ 1183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:42:53.224565 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:42:53.224565 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:42:53.224571 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:42:53.225884 (+ 1313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:53.226248 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:53.226249 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:53.226250 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:53.226307 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:53.226482 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:53.226483 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:53.226498 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:53.226677 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:42:53.226804 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:53.226977 (+ 173us) server_negotiation.cc:299] Negotiation successful
0619 02:42:53.227041 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 02:42:53.228211 21994 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45464:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:42:53.229801 21994 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:42:53.232376 22117 tablet_service.cc:1467] Processing CreateTablet for tablet 799490ce8bad48a083feacab1e134faf (DEFAULT_TABLE table=test-table [id=a940ec19bfab452182213d9deecc8b11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:42:53.232591 22117 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 799490ce8bad48a083feacab1e134faf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:42:53.234040 22075 tablet_bootstrap.cc:492] T 799490ce8bad48a083feacab1e134faf P 1c084879cb83420590952a75fa02ff81: Bootstrap starting.
I20260619 02:42:53.234352 22075 tablet_bootstrap.cc:654] T 799490ce8bad48a083feacab1e134faf P 1c084879cb83420590952a75fa02ff81: Neither blocks nor log segments found. Creating new log.
I20260619 02:42:53.235001 22075 tablet_bootstrap.cc:492] T 799490ce8bad48a083feacab1e134faf P 1c084879cb83420590952a75fa02ff81: No bootstrap required, opened a new log
I20260619 02:42:53.235054 22075 ts_tablet_manager.cc:1397] T 799490ce8bad48a083feacab1e134faf P 1c084879cb83420590952a75fa02ff81: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 02:42:53.235369 22075 raft_consensus.cc:348] T 799490ce8bad48a083feacab1e134faf P 1c084879cb83420590952a75fa02ff81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e6244cc32344d32ba646eb8c9a6f965" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36607 } } peers { permanent_uuid: "51b5ade6b85d4ab2976d15aea366ef09" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38187 } } peers { permanent_uuid: "1c084879cb83420590952a75fa02ff81" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 39547 } }
I20260619 02:42:53.235441 22075 raft_consensus.cc:374] T 799490ce8bad48a083feacab1e134faf P 1c084879cb83420590952a75fa02ff81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:42:53.235478 22075 raft_consensus.cc:729] T 799490ce8bad48a083feacab1e134faf P 1c084879cb83420590952a75fa02ff81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c084879cb83420590952a75fa02ff81, State: Initialized, Role: FOLLOWER
I20260619 02:42:53.235529 22075 consensus_queue.cc:260] T 799490ce8bad48a083feacab1e134faf P 1c084879cb83420590952a75fa02ff81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e6244cc32344d32ba646eb8c9a6f965" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36607 } } peers { permanent_uuid: "51b5ade6b85d4ab2976d15aea366ef09" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38187 } } peers { permanent_uuid: "1c084879cb83420590952a75fa02ff81" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 39547 } }
I20260619 02:42:53.235635 22075 ts_tablet_manager.cc:1428] T 799490ce8bad48a083feacab1e134faf P 1c084879cb83420590952a75fa02ff81: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260619 02:42:53.235893 22190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:53.231914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.66:36607 (local address 127.0.0.1:43378)
0619 02:42:53.232021 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:53.232033 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:53.232067 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:53.232392 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:53.232393 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:53.232395 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:42:53.232438 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:53.232440 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:53.233573 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:53.233574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:53.234659 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:53.234661 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:53.235621 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:53.235622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:53.235694 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:53.235742 (+ 48us) client_negotiation.cc:769] Sending connection context
0619 02:42:53.235776 (+ 34us) client_negotiation.cc:240] Negotiation successful
0619 02:42:53.235828 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"mutex_wait_us":39}
I20260619 02:42:53.236100 22460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:53.232017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43378 (local address 127.5.241.66:36607)
0619 02:42:53.232208 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:53.232214 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:53.232224 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:42:53.232271 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:53.232274 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:53.232291 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:53.232377 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:42:53.232578 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:53.233498 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:53.234743 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:53.235527 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:53.235861 (+ 334us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:53.235934 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:53.235961 (+ 27us) server_negotiation.cc:299] Negotiation successful
0619 02:42:53.236007 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":80,"threads_started":1}
I20260619 02:42:53.236881 22255 tablet_service.cc:1467] Processing CreateTablet for tablet 799490ce8bad48a083feacab1e134faf (DEFAULT_TABLE table=test-table [id=a940ec19bfab452182213d9deecc8b11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:42:53.237210 22255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 799490ce8bad48a083feacab1e134faf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:42:53.237749 22459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:53.231961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.67:38187 (local address 127.0.0.1:44522)
0619 02:42:53.232243 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:53.232253 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:53.232294 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:53.234316 (+ 2022us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:53.234317 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:53.234319 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:42:53.234354 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:53.234355 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:53.235589 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:53.235590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:53.236740 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:53.236744 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:53.237534 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:53.237535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:53.237597 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:53.237602 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 02:42:53.237651 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 02:42:53.237689 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":111,"threads_started":1}
I20260619 02:42:53.237986 22461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:53.232069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44522 (local address 127.5.241.67:38187)
0619 02:42:53.234064 (+ 1995us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:53.234069 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:53.234080 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:42:53.234118 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:53.234120 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:53.234135 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:53.234213 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:42:53.234424 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:53.235492 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:53.236840 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:53.237457 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:53.237534 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:53.237588 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:53.237865 (+ 277us) server_negotiation.cc:299] Negotiation successful
0619 02:42:53.237911 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1939,"thread_start_us":80,"threads_started":1}
I20260619 02:42:53.239517 22387 tablet_service.cc:1467] Processing CreateTablet for tablet 799490ce8bad48a083feacab1e134faf (DEFAULT_TABLE table=test-table [id=a940ec19bfab452182213d9deecc8b11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:42:53.239845 22387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 799490ce8bad48a083feacab1e134faf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:42:53.240607 22463 tablet_bootstrap.cc:492] T 799490ce8bad48a083feacab1e134faf P 1e6244cc32344d32ba646eb8c9a6f965: Bootstrap starting.
I20260619 02:42:53.241209 22463 tablet_bootstrap.cc:654] T 799490ce8bad48a083feacab1e134faf P 1e6244cc32344d32ba646eb8c9a6f965: Neither blocks nor log segments found. Creating new log.
I20260619 02:42:53.241737 22463 log.cc:824] T 799490ce8bad48a083feacab1e134faf P 1e6244cc32344d32ba646eb8c9a6f965: Log is configured to *not* fsync() on all Append() calls
I20260619 02:42:53.242487 22463 tablet_bootstrap.cc:492] T 799490ce8bad48a083feacab1e134faf P 1e6244cc32344d32ba646eb8c9a6f965: No bootstrap required, opened a new log
I20260619 02:42:53.242540 22463 ts_tablet_manager.cc:1397] T 799490ce8bad48a083feacab1e134faf P 1e6244cc32344d32ba646eb8c9a6f965: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:42:53.243023 22465 tablet_bootstrap.cc:492] T 799490ce8bad48a083feacab1e134faf P 51b5ade6b85d4ab2976d15aea366ef09: Bootstrap starting.
I20260619 02:42:53.243577 22465 tablet_bootstrap.cc:654] T 799490ce8bad48a083feacab1e134faf P 51b5ade6b85d4ab2976d15aea366ef09: Neither blocks nor log segments found. Creating new log.
I20260619 02:42:53.244179 22465 log.cc:824] T 799490ce8bad48a083feacab1e134faf P 51b5ade6b85d4ab2976d15aea366ef09: Log is configured to *not* fsync() on all Append() calls
I20260619 02:42:53.244349 22463 raft_consensus.cc:348] T 799490ce8bad48a083feacab1e134faf P 1e6244cc32344d32ba646eb8c9a6f965 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e6244cc32344d32ba646eb8c9a6f965" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36607 } } peers { permanent_uuid: "51b5ade6b85d4ab2976d15aea366ef09" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38187 } } peers { permanent_uuid: "1c084879cb83420590952a75fa02ff81" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 39547 } }
I20260619 02:42:53.244565 22463 raft_consensus.cc:374] T 799490ce8bad48a083feacab1e134faf P 1e6244cc32344d32ba646eb8c9a6f965 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:42:53.244609 22463 raft_consensus.cc:729] T 799490ce8bad48a083feacab1e134faf P 1e6244cc32344d32ba646eb8c9a6f965 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e6244cc32344d32ba646eb8c9a6f965, State: Initialized, Role: FOLLOWER
I20260619 02:42:53.244740 22463 consensus_queue.cc:260] T 799490ce8bad48a083feacab1e134faf P 1e6244cc32344d32ba646eb8c9a6f965 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e6244cc32344d32ba646eb8c9a6f965" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36607 } } peers { permanent_uuid: "51b5ade6b85d4ab2976d15aea366ef09" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38187 } } peers { permanent_uuid: "1c084879cb83420590952a75fa02ff81" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 39547 } }
I20260619 02:42:53.244979 22465 tablet_bootstrap.cc:492] T 799490ce8bad48a083feacab1e134faf P 51b5ade6b85d4ab2976d15aea366ef09: No bootstrap required, opened a new log
I20260619 02:42:53.245030 22465 ts_tablet_manager.cc:1397] T 799490ce8bad48a083feacab1e134faf P 51b5ade6b85d4ab2976d15aea366ef09: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:42:53.245060 22463 ts_tablet_manager.cc:1428] T 799490ce8bad48a083feacab1e134faf P 1e6244cc32344d32ba646eb8c9a6f965: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 02:42:53.245121 22321 heartbeater.cc:503] Master 127.5.241.126:39127 was elected leader, sending a full tablet report...
I20260619 02:42:53.246543 22465 raft_consensus.cc:348] T 799490ce8bad48a083feacab1e134faf P 51b5ade6b85d4ab2976d15aea366ef09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e6244cc32344d32ba646eb8c9a6f965" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36607 } } peers { permanent_uuid: "51b5ade6b85d4ab2976d15aea366ef09" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38187 } } peers { permanent_uuid: "1c084879cb83420590952a75fa02ff81" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 39547 } }
I20260619 02:42:53.246671 22465 raft_consensus.cc:374] T 799490ce8bad48a083feacab1e134faf P 51b5ade6b85d4ab2976d15aea366ef09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:42:53.246692 22465 raft_consensus.cc:729] T 799490ce8bad48a083feacab1e134faf P 51b5ade6b85d4ab2976d15aea366ef09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51b5ade6b85d4ab2976d15aea366ef09, State: Initialized, Role: FOLLOWER
I20260619 02:42:53.246826 22465 consensus_queue.cc:260] T 799490ce8bad48a083feacab1e134faf P 51b5ade6b85d4ab2976d15aea366ef09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e6244cc32344d32ba646eb8c9a6f965" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36607 } } peers { permanent_uuid: "51b5ade6b85d4ab2976d15aea366ef09" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38187 } } peers { permanent_uuid: "1c084879cb83420590952a75fa02ff81" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 39547 } }
I20260619 02:42:53.247110 22453 heartbeater.cc:503] Master 127.5.241.126:39127 was elected leader, sending a full tablet report...
I20260619 02:42:53.247097 22465 ts_tablet_manager.cc:1428] T 799490ce8bad48a083feacab1e134faf P 51b5ade6b85d4ab2976d15aea366ef09: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260619 02:42:53.331858 22322 tablet.cc:2369] T 799490ce8bad48a083feacab1e134faf P 1e6244cc32344d32ba646eb8c9a6f965: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:42:53.430998 22184 tablet.cc:2369] T 799490ce8bad48a083feacab1e134faf P 1c084879cb83420590952a75fa02ff81: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:42:53.452941 22454 tablet.cc:2369] T 799490ce8bad48a083feacab1e134faf P 51b5ade6b85d4ab2976d15aea366ef09: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:42:53.465693 22468 raft_consensus.cc:482] T 799490ce8bad48a083feacab1e134faf P 51b5ade6b85d4ab2976d15aea366ef09 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:42:53.465817 22468 raft_consensus.cc:504] T 799490ce8bad48a083feacab1e134faf P 51b5ade6b85d4ab2976d15aea366ef09 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e6244cc32344d32ba646eb8c9a6f965" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36607 } } peers { permanent_uuid: "51b5ade6b85d4ab2976d15aea366ef09" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38187 } } peers { permanent_uuid: "1c084879cb83420590952a75fa02ff81" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 39547 } }
I20260619 02:42:53.466231 22468 leader_election.cc:288] T 799490ce8bad48a083feacab1e134faf P 51b5ade6b85d4ab2976d15aea366ef09 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1e6244cc32344d32ba646eb8c9a6f965 (127.5.241.66:36607), 1c084879cb83420590952a75fa02ff81 (127.5.241.65:39547)
I20260619 02:42:53.469266 22460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:53.466534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:41903 (local address 127.5.241.66:36607)
0619 02:42:53.466590 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:53.466592 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:53.466604 (+ 12us) server_negotiation.cc:407] Connection header received
0619 02:42:53.466636 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:53.466637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:53.466647 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:53.466724 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:42:53.466965 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:53.467499 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:53.468457 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:53.468883 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:53.468936 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:53.468944 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:53.469152 (+ 208us) server_negotiation.cc:299] Negotiation successful
0619 02:42:53.469188 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 02:42:53.469254 22455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:53.466450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.66:36607 (local address 127.5.241.67:41903)
0619 02:42:53.466506 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:53.466519 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:53.466564 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:53.466762 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:53.466763 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:53.466783 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:42:53.466861 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:53.466863 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:53.467575 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:53.467576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:53.468367 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:53.468369 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:53.468968 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:53.468969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:53.469061 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:53.469066 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 02:42:53.469136 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 02:42:53.469194 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260619 02:42:53.469429 22469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:53.466570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:39547 (local address 127.5.241.67:44661)
0619 02:42:53.466719 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:53.466730 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:53.466815 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:53.466976 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:53.466977 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:53.466979 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:42:53.467047 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:53.467049 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:53.467737 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:53.467738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:53.468506 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:53.468508 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:53.469161 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:53.469161 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:53.469229 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:53.469233 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:42:53.469291 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 02:42:53.469343 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":99,"threads_started":1}
I20260619 02:42:53.469658 22470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:53.466687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:44661 (local address 127.5.241.65:39547)
0619 02:42:53.466819 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:53.466821 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:53.466831 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:42:53.466884 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:53.466884 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:53.466895 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:53.466966 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:42:53.467117 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:53.467622 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:53.468603 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:53.469071 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:53.469147 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:53.469157 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:53.469310 (+ 153us) server_negotiation.cc:299] Negotiation successful
0619 02:42:53.469587 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":53,"threads_started":1}
I20260619 02:42:53.469668 22275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "799490ce8bad48a083feacab1e134faf" candidate_uuid: "51b5ade6b85d4ab2976d15aea366ef09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e6244cc32344d32ba646eb8c9a6f965" is_pre_election: true
I20260619 02:42:53.469818 22275 raft_consensus.cc:2454] T 799490ce8bad48a083feacab1e134faf P 1e6244cc32344d32ba646eb8c9a6f965 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51b5ade6b85d4ab2976d15aea366ef09 in term 0.
I20260619 02:42:53.469970 22137 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "799490ce8bad48a083feacab1e134faf" candidate_uuid: "51b5ade6b85d4ab2976d15aea366ef09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c084879cb83420590952a75fa02ff81" is_pre_election: true
I20260619 02:42:53.470046 22342 leader_election.cc:302] T 799490ce8bad48a083feacab1e134faf P 51b5ade6b85d4ab2976d15aea366ef09 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e6244cc32344d32ba646eb8c9a6f965, 51b5ade6b85d4ab2976d15aea366ef09; no voters:
I20260619 02:42:53.470130 22137 raft_consensus.cc:2454] T 799490ce8bad48a083feacab1e134faf P 1c084879cb83420590952a75fa02ff81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51b5ade6b85d4ab2976d15aea366ef09 in term 0.
I20260619 02:42:53.470180 22468 raft_consensus.cc:2781] T 799490ce8bad48a083feacab1e134faf P 51b5ade6b85d4ab2976d15aea366ef09 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:42:53.470232 22468 raft_consensus.cc:482] T 799490ce8bad48a083feacab1e134faf P 51b5ade6b85d4ab2976d15aea366ef09 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:42:53.470257 22468 raft_consensus.cc:3037] T 799490ce8bad48a083feacab1e134faf P 51b5ade6b85d4ab2976d15aea366ef09 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:42:53.471040 22468 raft_consensus.cc:504] T 799490ce8bad48a083feacab1e134faf P 51b5ade6b85d4ab2976d15aea366ef09 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e6244cc32344d32ba646eb8c9a6f965" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36607 } } peers { permanent_uuid: "51b5ade6b85d4ab2976d15aea366ef09" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38187 } } peers { permanent_uuid: "1c084879cb83420590952a75fa02ff81" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 39547 } }
I20260619 02:42:53.471175 22468 leader_election.cc:288] T 799490ce8bad48a083feacab1e134faf P 51b5ade6b85d4ab2976d15aea366ef09 [CANDIDATE]: Term 1 election: Requested vote from peers 1e6244cc32344d32ba646eb8c9a6f965 (127.5.241.66:36607), 1c084879cb83420590952a75fa02ff81 (127.5.241.65:39547)
I20260619 02:42:53.471357 22275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "799490ce8bad48a083feacab1e134faf" candidate_uuid: "51b5ade6b85d4ab2976d15aea366ef09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e6244cc32344d32ba646eb8c9a6f965"
I20260619 02:42:53.471357 22137 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "799490ce8bad48a083feacab1e134faf" candidate_uuid: "51b5ade6b85d4ab2976d15aea366ef09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c084879cb83420590952a75fa02ff81"
I20260619 02:42:53.471438 22275 raft_consensus.cc:3037] T 799490ce8bad48a083feacab1e134faf P 1e6244cc32344d32ba646eb8c9a6f965 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:42:53.471436 22137 raft_consensus.cc:3037] T 799490ce8bad48a083feacab1e134faf P 1c084879cb83420590952a75fa02ff81 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:42:53.472043 22137 raft_consensus.cc:2454] T 799490ce8bad48a083feacab1e134faf P 1c084879cb83420590952a75fa02ff81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51b5ade6b85d4ab2976d15aea366ef09 in term 1.
I20260619 02:42:53.472218 22275 raft_consensus.cc:2454] T 799490ce8bad48a083feacab1e134faf P 1e6244cc32344d32ba646eb8c9a6f965 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51b5ade6b85d4ab2976d15aea366ef09 in term 1.
I20260619 02:42:53.472222 22342 leader_election.cc:302] T 799490ce8bad48a083feacab1e134faf P 51b5ade6b85d4ab2976d15aea366ef09 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c084879cb83420590952a75fa02ff81, 51b5ade6b85d4ab2976d15aea366ef09; no voters:
I20260619 02:42:53.472313 22468 raft_consensus.cc:2781] T 799490ce8bad48a083feacab1e134faf P 51b5ade6b85d4ab2976d15aea366ef09 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:42:53.472457 22468 raft_consensus.cc:686] T 799490ce8bad48a083feacab1e134faf P 51b5ade6b85d4ab2976d15aea366ef09 [term 1 LEADER]: Becoming Leader. State: Replica: 51b5ade6b85d4ab2976d15aea366ef09, State: Running, Role: LEADER
I20260619 02:42:53.472548 22468 consensus_queue.cc:237] T 799490ce8bad48a083feacab1e134faf P 51b5ade6b85d4ab2976d15aea366ef09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e6244cc32344d32ba646eb8c9a6f965" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36607 } } peers { permanent_uuid: "51b5ade6b85d4ab2976d15aea366ef09" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38187 } } peers { permanent_uuid: "1c084879cb83420590952a75fa02ff81" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 39547 } }
I20260619 02:42:53.473398 21993 catalog_manager.cc:5495] T 799490ce8bad48a083feacab1e134faf P 51b5ade6b85d4ab2976d15aea366ef09 reported cstate change: term changed from 0 to 1, leader changed from <none> to 51b5ade6b85d4ab2976d15aea366ef09 (127.5.241.67). New cstate: current_term: 1 leader_uuid: "51b5ade6b85d4ab2976d15aea366ef09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e6244cc32344d32ba646eb8c9a6f965" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51b5ade6b85d4ab2976d15aea366ef09" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38187 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c084879cb83420590952a75fa02ff81" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 39547 } health_report { overall_health: UNKNOWN } } }
I20260619 02:42:53.519035 22461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:53.516929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44528 (local address 127.5.241.67:38187)
0619 02:42:53.516997 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:53.516998 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:53.517007 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:42:53.517041 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:53.517042 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:53.517059 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:53.517129 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:42:53.517233 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:53.517699 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:53.518375 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:53.518548 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:53.518597 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:53.518737 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:42:53.518857 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:42:53.518917 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:53.518930 (+ 13us) server_negotiation.cc:299] Negotiation successful
0619 02:42:53.518945 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 02:42:53.520988 22275 raft_consensus.cc:1261] T 799490ce8bad48a083feacab1e134faf P 1e6244cc32344d32ba646eb8c9a6f965 [term 1 FOLLOWER]: Refusing update from remote peer 51b5ade6b85d4ab2976d15aea366ef09: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:42:53.521054 22137 raft_consensus.cc:1261] T 799490ce8bad48a083feacab1e134faf P 1c084879cb83420590952a75fa02ff81 [term 1 FOLLOWER]: Refusing update from remote peer 51b5ade6b85d4ab2976d15aea366ef09: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:42:53.521353 22471 consensus_queue.cc:1035] T 799490ce8bad48a083feacab1e134faf P 51b5ade6b85d4ab2976d15aea366ef09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e6244cc32344d32ba646eb8c9a6f965" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:42:53.521512 22468 consensus_queue.cc:1035] T 799490ce8bad48a083feacab1e134faf P 51b5ade6b85d4ab2976d15aea366ef09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c084879cb83420590952a75fa02ff81" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 39547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:42:53.523799 22473 mvcc.cc:205] Tried to move back new op lower bound from 7298404243539222528 to 7298404243344150528. Current Snapshot: MvccSnapshot[applied={T|T < 7298404243539222528}]
I20260619 02:42:53.524521 22475 mvcc.cc:205] Tried to move back new op lower bound from 7298404243539222528 to 7298404243344150528. Current Snapshot: MvccSnapshot[applied={T|T < 7298404243539222528 or (T in {7298404243539222528})}]
I20260619 02:42:53.526010 21993 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45464:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:42:53.526106 21993 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45464:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:42:53.526842 21993 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190: Sending DeleteTablet for 3 replicas of tablet 799490ce8bad48a083feacab1e134faf
I20260619 02:42:53.527181 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 22032
I20260619 02:42:53.527169 22387 tablet_service.cc:1514] Processing DeleteTablet for tablet 799490ce8bad48a083feacab1e134faf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:42:53 UTC) from {username='kudu', principal='kudu/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:44522
I20260619 02:42:53.527351 22117 tablet_service.cc:1514] Processing DeleteTablet for tablet 799490ce8bad48a083feacab1e134faf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:42:53 UTC) from {username='kudu', principal='kudu/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:51622
I20260619 02:42:53.527681 22255 tablet_service.cc:1514] Processing DeleteTablet for tablet 799490ce8bad48a083feacab1e134faf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:42:53 UTC) from {username='kudu', principal='kudu/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:43378
I20260619 02:42:53.527884 22483 tablet_replica.cc:330] T 799490ce8bad48a083feacab1e134faf P 1e6244cc32344d32ba646eb8c9a6f965: stopping tablet replica
I20260619 02:42:53.527890 22482 tablet_replica.cc:330] T 799490ce8bad48a083feacab1e134faf P 51b5ade6b85d4ab2976d15aea366ef09: stopping tablet replica
I20260619 02:42:53.527980 22482 raft_consensus.cc:2229] T 799490ce8bad48a083feacab1e134faf P 51b5ade6b85d4ab2976d15aea366ef09 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:42:53.528012 22483 raft_consensus.cc:2229] T 799490ce8bad48a083feacab1e134faf P 1e6244cc32344d32ba646eb8c9a6f965 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:42:53.528107 22482 raft_consensus.cc:2258] T 799490ce8bad48a083feacab1e134faf P 51b5ade6b85d4ab2976d15aea366ef09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:42:53.528220 22483 raft_consensus.cc:2258] T 799490ce8bad48a083feacab1e134faf P 1e6244cc32344d32ba646eb8c9a6f965 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:42:53.528560 22482 ts_tablet_manager.cc:1905] T 799490ce8bad48a083feacab1e134faf P 51b5ade6b85d4ab2976d15aea366ef09: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:42:53.528643 22483 ts_tablet_manager.cc:1905] T 799490ce8bad48a083feacab1e134faf P 1e6244cc32344d32ba646eb8c9a6f965: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:42:53.530053 22482 ts_tablet_manager.cc:1918] T 799490ce8bad48a083feacab1e134faf P 51b5ade6b85d4ab2976d15aea366ef09: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:42:53.530139 22482 log.cc:1196] T 799490ce8bad48a083feacab1e134faf P 51b5ade6b85d4ab2976d15aea366ef09: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-2/wal/wals/799490ce8bad48a083feacab1e134faf
I20260619 02:42:53.530146 22483 ts_tablet_manager.cc:1918] T 799490ce8bad48a083feacab1e134faf P 1e6244cc32344d32ba646eb8c9a6f965: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:42:53.530201 22483 log.cc:1196] T 799490ce8bad48a083feacab1e134faf P 1e6244cc32344d32ba646eb8c9a6f965: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-1/wal/wals/799490ce8bad48a083feacab1e134faf
I20260619 02:42:53.530432 22482 ts_tablet_manager.cc:1939] T 799490ce8bad48a083feacab1e134faf P 51b5ade6b85d4ab2976d15aea366ef09: Deleting consensus metadata
I20260619 02:42:53.530470 22483 ts_tablet_manager.cc:1939] T 799490ce8bad48a083feacab1e134faf P 1e6244cc32344d32ba646eb8c9a6f965: Deleting consensus metadata
I20260619 02:42:53.530912 21980 catalog_manager.cc:4841] TS 1e6244cc32344d32ba646eb8c9a6f965 (127.5.241.66:36607): tablet 799490ce8bad48a083feacab1e134faf (table test-table [id=a940ec19bfab452182213d9deecc8b11]) successfully deleted
I20260619 02:42:53.530929 21979 catalog_manager.cc:4841] TS 51b5ade6b85d4ab2976d15aea366ef09 (127.5.241.67:38187): tablet 799490ce8bad48a083feacab1e134faf (table test-table [id=a940ec19bfab452182213d9deecc8b11]) successfully deleted
W20260619 02:42:53.532083 21980 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.241.65:39547 (error 108)
I20260619 02:42:53.533445 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 22188
I20260619 02:42:53.533601 22459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:53.533454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:39547 (local address 127.0.0.1:51636)
0619 02:42:53.533522 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:53.533565 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.65:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260619 02:42:53.535380 21980 catalog_manager.cc:4584] TS 1c084879cb83420590952a75fa02ff81 (127.5.241.65:39547): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 799490ce8bad48a083feacab1e134faf: Network error: Client connection negotiation failed: client connection to 127.5.241.65:39547: connect: Connection refused (error 111)
I20260619 02:42:53.538725 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 22324
I20260619 02:42:53.544203 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 21961
I20260619 02:43:13.549697 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:39127
--webserver_interface=127.5.241.126
--webserver_port=34563
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:44479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:39127
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:43:13.631039 22485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:13.631285 22485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:13.631326 22485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:13.632918 22485 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 02:43:13.632977 22485 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 02:43:13.633001 22485 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 02:43:13.633023 22485 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:43:13.633066 22485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:13.633091 22485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:13.633127 22485 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:43:13.633157 22485 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:43:13.635390 22485 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:44479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:39127
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:39127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.241.126
--webserver_port=34563
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22485
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:43:13.635687 22485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:13.635946 22485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:13.641372 22490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:13.641427 22491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:13.641368 22493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:13.641464 22485 server_base.cc:1034] running on GCE node
I20260619 02:43:13.641820 22485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:13.642127 22485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:13.643271 22485 hybrid_clock.cc:648] HybridClock initialized: now 1781836993643247 us; error 38 us; skew 500 ppm
Jun 19 02:43:13 dist-test-slave-zpbg krb5kdc[21417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836993, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:13.648775 22485 init.cc:377] Logged in from keytab as kudu/127.5.241.126@KRBTEST.COM (short username kudu)
I20260619 02:43:13.649259 22485 webserver.cc:460] Webserver started at http://127.5.241.126:34563/ using document root <none> and password file <none>
I20260619 02:43:13.649492 22485 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:13.649562 22485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:13.650945 22485 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:43:13.651659 22500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:13.651849 22485 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:43:13.651948 22485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "6c315d868f78434a8e1c444c0279e190"
format_stamp: "Formatted at 2026-06-19 02:42:31 on dist-test-slave-zpbg"
server_key: "db9e53fe736cf602ca04bf799dbfb167"
server_key_iv: "135d364f0237f5781bdfbfb89800aee9"
server_key_version: "encryptionkey@0"
I20260619 02:43:13.652316 22485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:13.670481 22485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:13.672554 22485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:13.672719 22485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:13.676956 22485 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.126:39127
I20260619 02:43:13.676993 22552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.126:39127 every 8 connection(s)
I20260619 02:43:13.677395 22485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:43:13.678803 22553 sys_catalog.cc:263] Verifying existing consensus state
I20260619 02:43:13.679394 22553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190: Bootstrap starting.
I20260619 02:43:13.684897 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 22485
I20260619 02:43:13.685213 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.65:39547
--local_ip_for_outbound_sockets=127.5.241.65
--tserver_master_addrs=127.5.241.126:39127
--webserver_port=35705
--webserver_interface=127.5.241.65
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab
--principal=kudu/127.5.241.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:44479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:43:13.687628 22553 log.cc:824] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:13.689800 22553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:43:13.690104 22553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190: Bootstrap complete.
I20260619 02:43:13.692373 22553 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c315d868f78434a8e1c444c0279e190" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 39127 } }
I20260619 02:43:13.692615 22553 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c315d868f78434a8e1c444c0279e190, State: Initialized, Role: FOLLOWER
I20260619 02:43:13.692793 22553 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c315d868f78434a8e1c444c0279e190" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 39127 } }
I20260619 02:43:13.692871 22553 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:43:13.692922 22553 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:43:13.692970 22553 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190 [term 2 FOLLOWER]: Advancing to term 3
I20260619 02:43:13.693914 22553 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c315d868f78434a8e1c444c0279e190" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 39127 } }
I20260619 02:43:13.694029 22553 leader_election.cc:302] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c315d868f78434a8e1c444c0279e190; no voters:
I20260619 02:43:13.694226 22553 leader_election.cc:288] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260619 02:43:13.694262 22557 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 02:43:13.694377 22557 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190 [term 3 LEADER]: Becoming Leader. State: Replica: 6c315d868f78434a8e1c444c0279e190, State: Running, Role: LEADER
I20260619 02:43:13.694460 22557 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c315d868f78434a8e1c444c0279e190" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 39127 } }
I20260619 02:43:13.694486 22553 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:43:13.694729 22557 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c315d868f78434a8e1c444c0279e190. Latest consensus state: current_term: 3 leader_uuid: "6c315d868f78434a8e1c444c0279e190" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c315d868f78434a8e1c444c0279e190" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 39127 } } }
I20260619 02:43:13.694816 22557 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190 [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:13.695230 22558 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "6c315d868f78434a8e1c444c0279e190" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c315d868f78434a8e1c444c0279e190" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 39127 } } }
I20260619 02:43:13.695309 22558 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190 [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:13.695930 22561 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:43:13.696609 22561 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:43:13.696754 22561 catalog_manager.cc:1259] Loaded cluster ID: 21a3f16bf7204b8e8b7a4af68209b14f
I20260619 02:43:13.696805 22561 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:43:13.697580 22561 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:43:13.697762 22561 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190: Loaded TSK: 0
I20260619 02:43:13.697818 22561 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6c315d868f78434a8e1c444c0279e190: Loaded TSK: 1
F20260619 02:43:13.698022 22561 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f1fba3db9ad google::LogMessage::Fail() at ??:0
@ 0x7f1fba3de88a google::LogMessage::SendToLog() at ??:0
@ 0x7f1fba3db4de google::LogMessage::Flush() at ??:0
@ 0x7f1fba3dca19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f1fbcb48104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f1fbcb45d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f1fc23351cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f1fc233548e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f1fc2335594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f1fc22fbc48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f1fc2305ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f1fbc2a90d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f1fbc29e954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f1fbd1026db start_thread at ??:0
@ 0x7f1fb8ed961f clone at ??:0
W20260619 02:43:13.799856 22555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:13.800041 22555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:13.800063 22555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:13.801211 22555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:13.801251 22555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:13.801277 22555 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.65
I20260619 02:43:13.802726 22555 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:44479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab
--principal=kudu/127.5.241.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.65:39547
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.241.65
--webserver_port=35705
--tserver_master_addrs=127.5.241.126:39127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22555
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.65
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:43:13.803022 22555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:13.803259 22555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:13.808233 22595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:13.808269 22594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:13.808573 22555 server_base.cc:1034] running on GCE node
W20260619 02:43:13.808631 22597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:13.808879 22555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:13.809149 22555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:13.810287 22555 hybrid_clock.cc:648] HybridClock initialized: now 1781836993810273 us; error 31 us; skew 500 ppm
Jun 19 02:43:13 dist-test-slave-zpbg krb5kdc[21417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836993, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:13.812973 22555 init.cc:377] Logged in from keytab as kudu/127.5.241.65@KRBTEST.COM (short username kudu)
I20260619 02:43:13.813473 22555 webserver.cc:460] Webserver started at http://127.5.241.65:35705/ using document root <none> and password file <none>
I20260619 02:43:13.813726 22555 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:13.813798 22555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:13.815222 22555 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:43:13.815963 22604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:13.816190 22555 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:43:13.816298 22555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-0/wal
uuid: "1c084879cb83420590952a75fa02ff81"
format_stamp: "Formatted at 2026-06-19 02:42:32 on dist-test-slave-zpbg"
server_key: "de1ee857088991bde4e566a46099bd1f"
server_key_iv: "bed7176c6ba9e35235a3f6062a0957a7"
server_key_version: "encryptionkey@0"
I20260619 02:43:13.816622 22555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:13.824700 22555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:13.827001 22555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:13.827159 22555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:13.827378 22555 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:43:13.827802 22611 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 02:43:13.829761 22555 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 02:43:13.829845 22555 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 02:43:13.829903 22555 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 02:43:13.830521 22555 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 02:43:13.830612 22555 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 02:43:13.830644 22611 tablet_bootstrap.cc:492] T 799490ce8bad48a083feacab1e134faf P 1c084879cb83420590952a75fa02ff81: Bootstrap starting.
I20260619 02:43:13.837798 22555 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.65:39547
I20260619 02:43:13.837930 22718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.65:39547 every 8 connection(s)
I20260619 02:43:13.838215 22555 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
I20260619 02:43:13.840371 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 22555
I20260619 02:43:13.840689 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.66:36607
--local_ip_for_outbound_sockets=127.5.241.66
--tserver_master_addrs=127.5.241.126:39127
--webserver_port=44697
--webserver_interface=127.5.241.66
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab
--principal=kudu/127.5.241.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:44479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:43:13.841521 22611 log.cc:824] T 799490ce8bad48a083feacab1e134faf P 1c084879cb83420590952a75fa02ff81: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:13.843791 22611 tablet_bootstrap.cc:492] T 799490ce8bad48a083feacab1e134faf P 1c084879cb83420590952a75fa02ff81: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:43:13.844087 22611 tablet_bootstrap.cc:492] T 799490ce8bad48a083feacab1e134faf P 1c084879cb83420590952a75fa02ff81: Bootstrap complete.
I20260619 02:43:13.845084 22611 ts_tablet_manager.cc:1397] T 799490ce8bad48a083feacab1e134faf P 1c084879cb83420590952a75fa02ff81: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.004s
I20260619 02:43:13.846168 22611 raft_consensus.cc:348] T 799490ce8bad48a083feacab1e134faf P 1c084879cb83420590952a75fa02ff81 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e6244cc32344d32ba646eb8c9a6f965" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36607 } } peers { permanent_uuid: "51b5ade6b85d4ab2976d15aea366ef09" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38187 } } peers { permanent_uuid: "1c084879cb83420590952a75fa02ff81" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 39547 } }
I20260619 02:43:13.846421 22611 raft_consensus.cc:729] T 799490ce8bad48a083feacab1e134faf P 1c084879cb83420590952a75fa02ff81 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c084879cb83420590952a75fa02ff81, State: Initialized, Role: FOLLOWER
I20260619 02:43:13.846575 22611 consensus_queue.cc:260] T 799490ce8bad48a083feacab1e134faf P 1c084879cb83420590952a75fa02ff81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e6244cc32344d32ba646eb8c9a6f965" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36607 } } peers { permanent_uuid: "51b5ade6b85d4ab2976d15aea366ef09" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38187 } } peers { permanent_uuid: "1c084879cb83420590952a75fa02ff81" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 39547 } }
I20260619 02:43:13.846906 22611 ts_tablet_manager.cc:1428] T 799490ce8bad48a083feacab1e134faf P 1c084879cb83420590952a75fa02ff81: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
W20260619 02:43:13.928421 22721 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:43:13.838922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.65:57953)
0619 02:43:13.839106 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:13.839151 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:13.839609 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:13.928182 (+ 88573us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:39127: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":114,"threads_started":1}
W20260619 02:43:13.928725 22719 heartbeater.cc:644] Failed to heartbeat to 127.5.241.126:39127 (0 consecutive failures): Network error: Failed to ping master at 127.5.241.126:39127: Client connection negotiation failed: client connection to 127.5.241.126:39127: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 02:43:13.929116 22721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:13.928989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.65:52831)
0619 02:43:13.929043 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:13.929087 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:39127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260619 02:43:13.929546 22721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:13.929404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.65:40715)
0619 02:43:13.929484 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:13.929520 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:39127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260619 02:43:13.929733 22719 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 02:43:13.942345 22723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:13.942559 22723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:13.942584 22723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:13.943827 22723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:13.943877 22723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:13.943905 22723 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.66
I20260619 02:43:13.945386 22723 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:44479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab
--principal=kudu/127.5.241.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.66:36607
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.241.66
--webserver_port=44697
--tserver_master_addrs=127.5.241.126:39127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22723
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.66
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:43:13.945642 22723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:13.945887 22723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:13.950971 22729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:13.950961 22732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:13.950958 22730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:13.951102 22723 server_base.cc:1034] running on GCE node
I20260619 02:43:13.951390 22723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:13.951617 22723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:13.952766 22723 hybrid_clock.cc:648] HybridClock initialized: now 1781836993952755 us; error 32 us; skew 500 ppm
Jun 19 02:43:13 dist-test-slave-zpbg krb5kdc[21417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836993, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:13.955319 22723 init.cc:377] Logged in from keytab as kudu/127.5.241.66@KRBTEST.COM (short username kudu)
I20260619 02:43:13.955754 22723 webserver.cc:460] Webserver started at http://127.5.241.66:44697/ using document root <none> and password file <none>
I20260619 02:43:13.955935 22723 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:13.955976 22723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:13.957245 22723 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:43:13.957871 22739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:13.958230 22723 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 02:43:13.958360 22723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-1/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-1/wal
uuid: "1e6244cc32344d32ba646eb8c9a6f965"
format_stamp: "Formatted at 2026-06-19 02:42:32 on dist-test-slave-zpbg"
server_key: "3d080433e7293677d7af8ae265138acf"
server_key_iv: "8ad93a6546cc48599734ef16cf090ab5"
server_key_version: "encryptionkey@0"
I20260619 02:43:13.958724 22723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:13.973592 22723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:13.975776 22723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:13.975945 22723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:13.976184 22723 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:43:13.976547 22723 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:43:13.976601 22723 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:13.976641 22723 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:43:13.976670 22723 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:13.982803 22723 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.66:36607
I20260619 02:43:13.982841 22852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.66:36607 every 8 connection(s)
I20260619 02:43:13.983183 22723 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
I20260619 02:43:13.984248 22855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:13.983882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.66:34201)
0619 02:43:13.984032 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:13.984100 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:39127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":74,"threads_started":1}
W20260619 02:43:13.984670 22853 heartbeater.cc:644] Failed to heartbeat to 127.5.241.126:39127 (0 consecutive failures): Network error: Failed to ping master at 127.5.241.126:39127: Client connection negotiation failed: client connection to 127.5.241.126:39127: connect: Connection refused (error 111)
I20260619 02:43:13.984969 22855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:13.984882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.66:47525)
0619 02:43:13.984926 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:13.984952 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:39127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260619 02:43:13.985302 22855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:13.985227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.66:33799)
0619 02:43:13.985268 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:13.985289 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:39127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260619 02:43:13.985453 22853 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 02:43:13.986694 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 22723
I20260619 02:43:13.987027 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.67:38187
--local_ip_for_outbound_sockets=127.5.241.67
--tserver_master_addrs=127.5.241.126:39127
--webserver_port=35923
--webserver_interface=127.5.241.67
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab
--principal=kudu/127.5.241.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:44479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:43:14.069265 22856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:14.069478 22856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:14.069502 22856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:14.070721 22856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:14.070791 22856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:14.070853 22856 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.67
I20260619 02:43:14.072412 22856 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:44479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab
--principal=kudu/127.5.241.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.67:38187
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.241.67
--webserver_port=35923
--tserver_master_addrs=127.5.241.126:39127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22856
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.67
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:43:14.072677 22856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:14.072918 22856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:14.078035 22864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:14.078035 22861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:14.078099 22862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:14.078289 22856 server_base.cc:1034] running on GCE node
I20260619 02:43:14.078495 22856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:14.078745 22856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:14.079928 22856 hybrid_clock.cc:648] HybridClock initialized: now 1781836994079905 us; error 39 us; skew 500 ppm
Jun 19 02:43:14 dist-test-slave-zpbg krb5kdc[21417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836994, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:14.082491 22856 init.cc:377] Logged in from keytab as kudu/127.5.241.67@KRBTEST.COM (short username kudu)
I20260619 02:43:14.082998 22856 webserver.cc:460] Webserver started at http://127.5.241.67:35923/ using document root <none> and password file <none>
I20260619 02:43:14.083225 22856 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:14.083271 22856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:14.084554 22856 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:43:14.085196 22871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:14.085412 22856 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:43:14.085513 22856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-2/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-2/wal
uuid: "51b5ade6b85d4ab2976d15aea366ef09"
format_stamp: "Formatted at 2026-06-19 02:42:32 on dist-test-slave-zpbg"
server_key: "78c5584faf61b792b41bd0209d810b76"
server_key_iv: "855742e05241d138923a434f6ef22ed2"
server_key_version: "encryptionkey@0"
I20260619 02:43:14.085852 22856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:14.100871 22856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:14.103204 22856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:14.103368 22856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:14.103618 22856 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:43:14.103930 22856 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:43:14.103984 22856 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:14.104027 22856 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:43:14.104055 22856 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:14.109972 22856 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.67:38187
I20260619 02:43:14.110023 22984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.67:38187 every 8 connection(s)
I20260619 02:43:14.110350 22856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
I20260619 02:43:14.111413 22987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:14.111006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.67:46219)
0619 02:43:14.111178 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:14.111251 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:39127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":63,"threads_started":1}
W20260619 02:43:14.111840 22985 heartbeater.cc:644] Failed to heartbeat to 127.5.241.126:39127 (0 consecutive failures): Network error: Failed to ping master at 127.5.241.126:39127: Client connection negotiation failed: client connection to 127.5.241.126:39127: connect: Connection refused (error 111)
I20260619 02:43:14.112118 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 22856
I20260619 02:43:14.112161 22987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:14.112038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.67:49497)
0619 02:43:14.112089 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:14.112123 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:39127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260619 02:43:14.112581 22987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:14.112454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.67:41129)
0619 02:43:14.112531 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:14.112560 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:39127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260619 02:43:14.112730 22985 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 02:43:14.930382 22989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:14.930144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.65:50633)
0619 02:43:14.930289 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:14.930327 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:39127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":78,"threads_started":1}
I20260619 02:43:14.985997 22990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:14.985782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.66:42167)
0619 02:43:14.985904 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:14.985940 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:39127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":56,"threads_started":1}
I20260619 02:43:15.113296 22991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:15.113081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.67:47187)
0619 02:43:15.113208 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:15.113248 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:39127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":59,"threads_started":1}
I20260619 02:43:15.672335 22992 raft_consensus.cc:482] T 799490ce8bad48a083feacab1e134faf P 1c084879cb83420590952a75fa02ff81 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:43:15.672540 22992 raft_consensus.cc:504] T 799490ce8bad48a083feacab1e134faf P 1c084879cb83420590952a75fa02ff81 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e6244cc32344d32ba646eb8c9a6f965" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36607 } } peers { permanent_uuid: "51b5ade6b85d4ab2976d15aea366ef09" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38187 } } peers { permanent_uuid: "1c084879cb83420590952a75fa02ff81" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 39547 } }
I20260619 02:43:15.672928 22992 leader_election.cc:288] T 799490ce8bad48a083feacab1e134faf P 1c084879cb83420590952a75fa02ff81 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1e6244cc32344d32ba646eb8c9a6f965 (127.5.241.66:36607), 51b5ade6b85d4ab2976d15aea366ef09 (127.5.241.67:38187)
Jun 19 02:43:15 dist-test-slave-zpbg krb5kdc[21417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836993, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.65@KRBTEST.COM for kudu/127.5.241.66@KRBTEST.COM
Jun 19 02:43:15 dist-test-slave-zpbg krb5kdc[21417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836993, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.65@KRBTEST.COM for kudu/127.5.241.67@KRBTEST.COM
I20260619 02:43:15.683663 22995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:15.673254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:60753 (local address 127.5.241.66:36607)
0619 02:43:15.673426 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:15.673435 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:15.673514 (+ 79us) server_negotiation.cc:407] Connection header received
0619 02:43:15.673978 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:15.673980 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:15.674017 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:15.674169 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:15.674899 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:15.675815 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:15.676767 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:15.677164 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:15.677282 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:15.679745 (+ 2463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:15.679750 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:15.679750 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:15.679782 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:15.681724 (+ 1942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:15.682195 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:15.682215 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:15.682216 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:15.682319 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:15.682583 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:15.682584 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:15.682584 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:15.682880 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:15.682982 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:15.683268 (+ 286us) server_negotiation.cc:299] Negotiation successful
0619 02:43:15.683546 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":46,"threads_started":1}
I20260619 02:43:15.683727 22996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:15.673250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:34057 (local address 127.5.241.67:38187)
0619 02:43:15.673381 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:15.673394 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:15.673419 (+ 25us) server_negotiation.cc:407] Connection header received
0619 02:43:15.673926 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:15.673928 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:15.673980 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:15.674141 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:15.674906 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:15.675773 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:15.676760 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:15.677150 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:15.677284 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:15.680058 (+ 2774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:15.680063 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:15.680063 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:15.680076 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:15.682030 (+ 1954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:15.682390 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:15.682392 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:15.682393 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:15.682487 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:15.682691 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:15.682692 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:15.682692 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:15.682972 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:15.683063 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:15.683269 (+ 206us) server_negotiation.cc:299] Negotiation successful
0619 02:43:15.683542 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":62,"threads_started":1}
I20260619 02:43:15.683671 22993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:15.673136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.66:36607 (local address 127.5.241.65:60753)
0619 02:43:15.673443 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:15.673452 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:15.673492 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:15.674193 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:15.674196 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:15.674572 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:43:15.674744 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:15.674750 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:15.675908 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:15.675909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:15.676661 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:15.676663 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:15.677254 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:15.677255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:15.677386 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:15.678009 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:43:15.678013 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:43:15.679576 (+ 1563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:43:15.681821 (+ 2245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:15.681827 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:15.681829 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:15.682051 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:15.682395 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:15.682396 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:15.682396 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:15.682505 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:15.682960 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:43:15.682967 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:43:15.683091 (+ 124us) client_negotiation.cc:769] Sending connection context
0619 02:43:15.683200 (+ 109us) client_negotiation.cc:240] Negotiation successful
0619 02:43:15.683573 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"mutex_wait_us":49,"thread_start_us":54,"threads_started":1}
I20260619 02:43:15.684091 22994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:15.673110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.67:38187 (local address 127.5.241.65:34057)
0619 02:43:15.673271 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:15.673284 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:15.673359 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:15.674194 (+ 835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:15.674196 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:15.674571 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:43:15.674744 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:15.674750 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:15.675879 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:15.675880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:15.676667 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:15.676669 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:15.677261 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:15.677262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:15.677385 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:15.677956 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:43:15.677964 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:43:15.679946 (+ 1982us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:43:15.682113 (+ 2167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:15.682114 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:15.682115 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:15.682295 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:15.682558 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:15.682559 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:15.682559 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:15.682627 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:15.683045 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:43:15.683046 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:43:15.683109 (+ 63us) client_negotiation.cc:769] Sending connection context
0619 02:43:15.683203 (+ 94us) client_negotiation.cc:240] Negotiation successful
0619 02:43:15.683679 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":321,"thread_start_us":59,"threads_started":1}
W20260619 02:43:15.684873 22607 leader_election.cc:341] T 799490ce8bad48a083feacab1e134faf P 1c084879cb83420590952a75fa02ff81 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1e6244cc32344d32ba646eb8c9a6f965 (127.5.241.66:36607): Not found: Tablet not found: 799490ce8bad48a083feacab1e134faf
W20260619 02:43:15.684955 22606 leader_election.cc:341] T 799490ce8bad48a083feacab1e134faf P 1c084879cb83420590952a75fa02ff81 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 51b5ade6b85d4ab2976d15aea366ef09 (127.5.241.67:38187): Not found: Tablet not found: 799490ce8bad48a083feacab1e134faf
I20260619 02:43:15.685022 22606 leader_election.cc:302] T 799490ce8bad48a083feacab1e134faf P 1c084879cb83420590952a75fa02ff81 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1c084879cb83420590952a75fa02ff81; no voters: 1e6244cc32344d32ba646eb8c9a6f965, 51b5ade6b85d4ab2976d15aea366ef09
I20260619 02:43:15.685124 22992 raft_consensus.cc:2726] T 799490ce8bad48a083feacab1e134faf P 1c084879cb83420590952a75fa02ff81 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 02:43:15.931102 22994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:15.930935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.65:45371)
0619 02:43:15.931018 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:15.931062 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:39127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260619 02:43:15.986730 22997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:15.986479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.66:43255)
0619 02:43:15.986626 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:15.986681 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:39127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":61,"threads_started":1}
I20260619 02:43:16.114125 22998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:16.113878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.67:38943)
0619 02:43:16.114023 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:16.114072 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:39127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":70,"threads_started":1}
I20260619 02:43:16.931958 22999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:16.931714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.65:33457)
0619 02:43:16.931873 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:16.931912 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:39127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":75,"threads_started":1}
I20260619 02:43:16.987424 23000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:16.987185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.66:53427)
0619 02:43:16.987312 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:16.987360 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:39127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":56,"threads_started":1}
I20260619 02:43:17.114957 23001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:17.114638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.67:48409)
0619 02:43:17.114873 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:17.114912 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:39127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":99,"threads_started":1}
I20260619 02:43:17.580830 23002 raft_consensus.cc:482] T 799490ce8bad48a083feacab1e134faf P 1c084879cb83420590952a75fa02ff81 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:43:17.580936 23002 raft_consensus.cc:504] T 799490ce8bad48a083feacab1e134faf P 1c084879cb83420590952a75fa02ff81 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e6244cc32344d32ba646eb8c9a6f965" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36607 } } peers { permanent_uuid: "51b5ade6b85d4ab2976d15aea366ef09" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38187 } } peers { permanent_uuid: "1c084879cb83420590952a75fa02ff81" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 39547 } }
I20260619 02:43:17.581137 23002 leader_election.cc:288] T 799490ce8bad48a083feacab1e134faf P 1c084879cb83420590952a75fa02ff81 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1e6244cc32344d32ba646eb8c9a6f965 (127.5.241.66:36607), 51b5ade6b85d4ab2976d15aea366ef09 (127.5.241.67:38187)
W20260619 02:43:17.581588 22606 leader_election.cc:341] T 799490ce8bad48a083feacab1e134faf P 1c084879cb83420590952a75fa02ff81 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 51b5ade6b85d4ab2976d15aea366ef09 (127.5.241.67:38187): Not found: Tablet not found: 799490ce8bad48a083feacab1e134faf
W20260619 02:43:17.581687 22607 leader_election.cc:341] T 799490ce8bad48a083feacab1e134faf P 1c084879cb83420590952a75fa02ff81 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1e6244cc32344d32ba646eb8c9a6f965 (127.5.241.66:36607): Not found: Tablet not found: 799490ce8bad48a083feacab1e134faf
I20260619 02:43:17.581822 22607 leader_election.cc:302] T 799490ce8bad48a083feacab1e134faf P 1c084879cb83420590952a75fa02ff81 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1c084879cb83420590952a75fa02ff81; no voters: 1e6244cc32344d32ba646eb8c9a6f965, 51b5ade6b85d4ab2976d15aea366ef09
I20260619 02:43:17.581907 23002 raft_consensus.cc:2726] T 799490ce8bad48a083feacab1e134faf P 1c084879cb83420590952a75fa02ff81 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 02:43:17.932715 23003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:17.932462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.65:46457)
0619 02:43:17.932619 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:17.932655 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:39127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":64,"threads_started":1}
I20260619 02:43:17.988111 23004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:17.987831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.66:46021)
0619 02:43:17.987959 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:17.987994 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:39127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":70,"threads_started":1}
I20260619 02:43:18.115665 23005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:18.115405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.67:50133)
0619 02:43:18.115546 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:18.115584 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:39127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":66,"threads_started":1}
I20260619 02:43:18.933429 23006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:18.933229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.65:46839)
0619 02:43:18.933350 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:18.933383 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:39127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":59,"threads_started":1}
I20260619 02:43:18.988821 23007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:18.988551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.66:57241)
0619 02:43:18.988716 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:18.988753 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:39127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":64,"threads_started":1}
I20260619 02:43:19.116472 23008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:19.116193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.67:34483)
0619 02:43:19.116363 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:19.116411 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:39127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":64,"threads_started":1}
I20260619 02:43:19.934163 23009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:19.933893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.65:33227)
0619 02:43:19.934060 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:19.934113 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:39127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":79,"threads_started":1}
I20260619 02:43:19.989452 23010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:19.989247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.66:40771)
0619 02:43:19.989371 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:19.989409 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:39127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":68,"threads_started":1}
I20260619 02:43:20.117291 23011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:20.117058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.67:58819)
0619 02:43:20.117205 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:20.117245 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:39127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":74,"threads_started":1}
I20260619 02:43:20.657931 23012 raft_consensus.cc:482] T 799490ce8bad48a083feacab1e134faf P 1c084879cb83420590952a75fa02ff81 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:43:20.658037 23012 raft_consensus.cc:504] T 799490ce8bad48a083feacab1e134faf P 1c084879cb83420590952a75fa02ff81 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e6244cc32344d32ba646eb8c9a6f965" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36607 } } peers { permanent_uuid: "51b5ade6b85d4ab2976d15aea366ef09" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38187 } } peers { permanent_uuid: "1c084879cb83420590952a75fa02ff81" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 39547 } }
I20260619 02:43:20.658232 23012 leader_election.cc:288] T 799490ce8bad48a083feacab1e134faf P 1c084879cb83420590952a75fa02ff81 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1e6244cc32344d32ba646eb8c9a6f965 (127.5.241.66:36607), 51b5ade6b85d4ab2976d15aea366ef09 (127.5.241.67:38187)
W20260619 02:43:20.658758 22606 leader_election.cc:341] T 799490ce8bad48a083feacab1e134faf P 1c084879cb83420590952a75fa02ff81 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 51b5ade6b85d4ab2976d15aea366ef09 (127.5.241.67:38187): Not found: Tablet not found: 799490ce8bad48a083feacab1e134faf
W20260619 02:43:20.658864 22607 leader_election.cc:341] T 799490ce8bad48a083feacab1e134faf P 1c084879cb83420590952a75fa02ff81 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1e6244cc32344d32ba646eb8c9a6f965 (127.5.241.66:36607): Not found: Tablet not found: 799490ce8bad48a083feacab1e134faf
I20260619 02:43:20.658897 22607 leader_election.cc:302] T 799490ce8bad48a083feacab1e134faf P 1c084879cb83420590952a75fa02ff81 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1c084879cb83420590952a75fa02ff81; no voters: 1e6244cc32344d32ba646eb8c9a6f965, 51b5ade6b85d4ab2976d15aea366ef09
I20260619 02:43:20.658979 23012 raft_consensus.cc:2726] T 799490ce8bad48a083feacab1e134faf P 1c084879cb83420590952a75fa02ff81 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 02:43:20.934965 23013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:20.934717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.65:52207)
0619 02:43:20.934875 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:20.934918 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:39127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":81,"threads_started":1}
I20260619 02:43:20.990134 23014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:20.989921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.66:39511)
0619 02:43:20.990051 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:20.990085 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:39127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":69,"threads_started":1}
I20260619 02:43:21.118000 23015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:21.117770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.67:56153)
0619 02:43:21.117919 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:21.117956 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:39127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":64,"threads_started":1}
I20260619 02:43:21.935679 23016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:21.935425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.65:38789)
0619 02:43:21.935584 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:21.935624 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:39127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":74,"threads_started":1}
I20260619 02:43:21.990837 23017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:21.990612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.66:60569)
0619 02:43:21.990738 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:21.990796 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:39127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":57,"threads_started":1}
I20260619 02:43:22.118831 23018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:22.118521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.67:43387)
0619 02:43:22.118681 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:22.118721 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:39127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":80,"threads_started":1}
I20260619 02:43:22.936472 23019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:22.936258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.65:52063)
0619 02:43:22.936389 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:22.936427 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:39127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":70,"threads_started":1}
I20260619 02:43:22.991612 23020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:22.991345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.66:55125)
0619 02:43:22.991507 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:22.991557 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:39127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":73,"threads_started":1}
I20260619 02:43:23.119721 23021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:23.119352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.67:35397)
0619 02:43:23.119513 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:23.119615 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:39127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":62,"threads_started":1}
I20260619 02:43:23.937202 23022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:23.936959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.65:59817)
0619 02:43:23.937115 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:23.937154 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:39127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":62,"threads_started":1}
I20260619 02:43:23.992317 23023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:23.992113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.66:45695)
0619 02:43:23.992241 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:23.992275 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:39127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":69,"threads_started":1}
I20260619 02:43:24.120573 23024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:24.120281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.67:33835)
0619 02:43:24.120469 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:24.120518 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:39127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":85,"threads_started":1}
I20260619 02:43:24.938107 23025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:24.937820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.65:34321)
0619 02:43:24.938013 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:24.938051 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:39127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":90,"threads_started":1}
I20260619 02:43:24.992980 23026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:24.992772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.66:45761)
0619 02:43:24.992902 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:24.992937 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:39127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":70,"threads_started":1}
I20260619 02:43:25.121366 23027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:25.121116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.67:32887)
0619 02:43:25.121264 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:25.121308 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:39127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":69,"threads_started":1}
I20260619 02:43:25.204257 23028 raft_consensus.cc:482] T 799490ce8bad48a083feacab1e134faf P 1c084879cb83420590952a75fa02ff81 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:43:25.204360 23028 raft_consensus.cc:504] T 799490ce8bad48a083feacab1e134faf P 1c084879cb83420590952a75fa02ff81 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e6244cc32344d32ba646eb8c9a6f965" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 36607 } } peers { permanent_uuid: "51b5ade6b85d4ab2976d15aea366ef09" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38187 } } peers { permanent_uuid: "1c084879cb83420590952a75fa02ff81" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 39547 } }
I20260619 02:43:25.204553 23028 leader_election.cc:288] T 799490ce8bad48a083feacab1e134faf P 1c084879cb83420590952a75fa02ff81 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1e6244cc32344d32ba646eb8c9a6f965 (127.5.241.66:36607), 51b5ade6b85d4ab2976d15aea366ef09 (127.5.241.67:38187)
W20260619 02:43:25.204988 22607 leader_election.cc:341] T 799490ce8bad48a083feacab1e134faf P 1c084879cb83420590952a75fa02ff81 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1e6244cc32344d32ba646eb8c9a6f965 (127.5.241.66:36607): Not found: Tablet not found: 799490ce8bad48a083feacab1e134faf
W20260619 02:43:25.205080 22606 leader_election.cc:341] T 799490ce8bad48a083feacab1e134faf P 1c084879cb83420590952a75fa02ff81 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 51b5ade6b85d4ab2976d15aea366ef09 (127.5.241.67:38187): Not found: Tablet not found: 799490ce8bad48a083feacab1e134faf
I20260619 02:43:25.205139 22606 leader_election.cc:302] T 799490ce8bad48a083feacab1e134faf P 1c084879cb83420590952a75fa02ff81 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1c084879cb83420590952a75fa02ff81; no voters: 1e6244cc32344d32ba646eb8c9a6f965, 51b5ade6b85d4ab2976d15aea366ef09
I20260619 02:43:25.205232 23028 raft_consensus.cc:2726] T 799490ce8bad48a083feacab1e134faf P 1c084879cb83420590952a75fa02ff81 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 02:43:25.938989 23029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:25.938703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.65:46759)
0619 02:43:25.938894 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:25.938941 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:39127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":134,"threads_started":1}
I20260619 02:43:25.993750 23030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:25.993506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.66:50069)
0619 02:43:25.993660 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:25.993703 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:39127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":66,"threads_started":1}
I20260619 02:43:26.122099 23031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:26.121871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.67:33487)
0619 02:43:26.122016 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:26.122050 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:39127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":90,"threads_started":1}
I20260619 02:43:26.939860 23032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:26.939626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.65:57577)
0619 02:43:26.939768 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:26.939812 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:39127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":79,"threads_started":1}
I20260619 02:43:26.994513 23033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:26.994284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.66:55781)
0619 02:43:26.994428 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:26.994469 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:39127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":73,"threads_started":1}
I20260619 02:43:27.122922 23034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:27.122648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.67:33833)
0619 02:43:27.122831 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:27.122875 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:39127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":75,"threads_started":1}
I20260619 02:43:27.940666 23035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:27.940411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.65:42321)
0619 02:43:27.940570 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:27.940616 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:39127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":64,"threads_started":1}
I20260619 02:43:27.995329 23036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:27.995057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.66:41825)
0619 02:43:27.995213 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:27.995261 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:39127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":79,"threads_started":1}
I20260619 02:43:28.123556 23037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:28.123341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.67:39529)
0619 02:43:28.123474 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:28.123510 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:39127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":73,"threads_started":1}
I20260619 02:43:28.941596 23038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:28.941323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.65:56031)
0619 02:43:28.941494 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:28.941538 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:39127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":73,"threads_started":1}
I20260619 02:43:28.996134 23039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:28.995908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:39127 (local address 127.5.241.66:49591)
0619 02:43:28.996032 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:28.996073 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.126:39127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":56,"threads_started":1}
I20260619 02:43:29.114259 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 22555
I20260619 02:43:29.119025 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 22723
I20260619 02:43:29.123435 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 22856
2026-06-19T02:43:29Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58398 ms)
[----------] 30 tests from SecurityITest (236278 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:43:29 dist-test-slave-zpbg krb5kdc[23042](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:43:29 dist-test-slave-zpbg krb5kdc[23042](info): set up 2 sockets
Jun 19 02:43:29 dist-test-slave-zpbg krb5kdc[23042](info): commencing operation
krb5kdc: starting...
W20260619 02:43:30.166435 6085 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Jun 19 02:43:30 dist-test-slave-zpbg krb5kdc[23042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:43:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:43:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.241.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
I20260619 02:43:30.287905 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:40231
--webserver_interface=127.5.241.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:36635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:40231
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:43:30.369975 23055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:30.370194 23055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:30.370218 23055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:30.371446 23055 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:43:30.371487 23055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:30.371502 23055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:30.371515 23055 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:43:30.371570 23055 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:43:30.373003 23055 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:36635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:40231
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:40231
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.241.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23055
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:43:30.373253 23055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:30.373478 23055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:30.378281 23061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:30.378319 23060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:30.378334 23055 server_base.cc:1034] running on GCE node
W20260619 02:43:30.378301 23063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:30.378804 23055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:30.379102 23055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:30.380262 23055 hybrid_clock.cc:648] HybridClock initialized: now 1781837010380251 us; error 32 us; skew 500 ppm
Jun 19 02:43:30 dist-test-slave-zpbg krb5kdc[23042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837010, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:30.382715 23055 init.cc:377] Logged in from keytab as kudu/127.5.241.126@KRBTEST.COM (short username kudu)
I20260619 02:43:30.383159 23055 webserver.cc:460] Webserver started at http://127.5.241.126:41481/ using document root <none> and password file <none>
I20260619 02:43:30.383334 23055 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:30.383375 23055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:30.383459 23055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:43:30.384419 23055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/master-0/data/instance:
uuid: "f4d4c49ba53e4e67b4d83a85f89e7cba"
format_stamp: "Formatted at 2026-06-19 02:43:30 on dist-test-slave-zpbg"
server_key: "6526d104e3bf0b480645b588e239e614"
server_key_iv: "ce56eefbbc21a8947f3f06e06299ac2c"
server_key_version: "encryptionkey@0"
I20260619 02:43:30.384707 23055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/master-0/wal/instance:
uuid: "f4d4c49ba53e4e67b4d83a85f89e7cba"
format_stamp: "Formatted at 2026-06-19 02:43:30 on dist-test-slave-zpbg"
server_key: "6526d104e3bf0b480645b588e239e614"
server_key_iv: "ce56eefbbc21a8947f3f06e06299ac2c"
server_key_version: "encryptionkey@0"
I20260619 02:43:30.385874 23055 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:43:30.386516 23070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:30.386703 23055 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:43:30.386817 23055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "f4d4c49ba53e4e67b4d83a85f89e7cba"
format_stamp: "Formatted at 2026-06-19 02:43:30 on dist-test-slave-zpbg"
server_key: "6526d104e3bf0b480645b588e239e614"
server_key_iv: "ce56eefbbc21a8947f3f06e06299ac2c"
server_key_version: "encryptionkey@0"
I20260619 02:43:30.386900 23055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:30.396505 23055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:30.398561 23055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:30.398720 23055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:30.402858 23055 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.126:40231
I20260619 02:43:30.402906 23122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.126:40231 every 8 connection(s)
I20260619 02:43:30.403260 23055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:43:30.404098 23123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:30.406514 23123 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4d4c49ba53e4e67b4d83a85f89e7cba: Bootstrap starting.
I20260619 02:43:30.407150 23123 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f4d4c49ba53e4e67b4d83a85f89e7cba: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:30.407469 23123 log.cc:824] T 00000000000000000000000000000000 P f4d4c49ba53e4e67b4d83a85f89e7cba: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:30.408160 23123 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4d4c49ba53e4e67b4d83a85f89e7cba: No bootstrap required, opened a new log
I20260619 02:43:30.409732 23123 raft_consensus.cc:348] T 00000000000000000000000000000000 P f4d4c49ba53e4e67b4d83a85f89e7cba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4d4c49ba53e4e67b4d83a85f89e7cba" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 40231 } }
I20260619 02:43:30.409868 23123 raft_consensus.cc:374] T 00000000000000000000000000000000 P f4d4c49ba53e4e67b4d83a85f89e7cba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:30.409914 23123 raft_consensus.cc:729] T 00000000000000000000000000000000 P f4d4c49ba53e4e67b4d83a85f89e7cba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4d4c49ba53e4e67b4d83a85f89e7cba, State: Initialized, Role: FOLLOWER
I20260619 02:43:30.410063 23123 consensus_queue.cc:260] T 00000000000000000000000000000000 P f4d4c49ba53e4e67b4d83a85f89e7cba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4d4c49ba53e4e67b4d83a85f89e7cba" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 40231 } }
I20260619 02:43:30.410123 23123 raft_consensus.cc:388] T 00000000000000000000000000000000 P f4d4c49ba53e4e67b4d83a85f89e7cba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:43:30.410144 23123 raft_consensus.cc:482] T 00000000000000000000000000000000 P f4d4c49ba53e4e67b4d83a85f89e7cba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:43:30.410214 23123 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f4d4c49ba53e4e67b4d83a85f89e7cba [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:30.410738 23123 raft_consensus.cc:504] T 00000000000000000000000000000000 P f4d4c49ba53e4e67b4d83a85f89e7cba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4d4c49ba53e4e67b4d83a85f89e7cba" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 40231 } }
I20260619 02:43:30.410904 23123 leader_election.cc:302] T 00000000000000000000000000000000 P f4d4c49ba53e4e67b4d83a85f89e7cba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f4d4c49ba53e4e67b4d83a85f89e7cba; no voters:
I20260619 02:43:30.411083 23123 leader_election.cc:288] T 00000000000000000000000000000000 P f4d4c49ba53e4e67b4d83a85f89e7cba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:43:30.411146 23126 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f4d4c49ba53e4e67b4d83a85f89e7cba [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:43:30.411326 23126 raft_consensus.cc:686] T 00000000000000000000000000000000 P f4d4c49ba53e4e67b4d83a85f89e7cba [term 1 LEADER]: Becoming Leader. State: Replica: f4d4c49ba53e4e67b4d83a85f89e7cba, State: Running, Role: LEADER
I20260619 02:43:30.411341 23123 sys_catalog.cc:564] T 00000000000000000000000000000000 P f4d4c49ba53e4e67b4d83a85f89e7cba [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:43:30.411424 23126 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4d4c49ba53e4e67b4d83a85f89e7cba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4d4c49ba53e4e67b4d83a85f89e7cba" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 40231 } }
I20260619 02:43:30.411790 23128 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4d4c49ba53e4e67b4d83a85f89e7cba [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4d4c49ba53e4e67b4d83a85f89e7cba. Latest consensus state: current_term: 1 leader_uuid: "f4d4c49ba53e4e67b4d83a85f89e7cba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4d4c49ba53e4e67b4d83a85f89e7cba" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 40231 } } }
I20260619 02:43:30.411868 23128 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4d4c49ba53e4e67b4d83a85f89e7cba [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:30.411931 23127 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4d4c49ba53e4e67b4d83a85f89e7cba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f4d4c49ba53e4e67b4d83a85f89e7cba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4d4c49ba53e4e67b4d83a85f89e7cba" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 40231 } } }
I20260619 02:43:30.411991 23127 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4d4c49ba53e4e67b4d83a85f89e7cba [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:30.412380 23134 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:43:30.412891 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 23055
I20260619 02:43:30.412936 23134 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:43:30.412997 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/master-0/wal/instance
I20260619 02:43:30.413211 6085 external_mini_cluster.cc:1428] Setting key 4f0cfb2ec99521622c6f9fa2c813cc3e
I20260619 02:43:30.414358 23134 catalog_manager.cc:1347] Generated new cluster ID: d7451ce8a58249879181b979a7460386
I20260619 02:43:30.414408 23134 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 02:43:30 dist-test-slave-zpbg krb5kdc[23042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:43:30.424814 23134 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:43:30.425316 23145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:30.413679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58602 (local address 127.5.241.126:40231)
0619 02:43:30.414059 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:30.414068 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:30.414101 (+ 33us) server_negotiation.cc:407] Connection header received
0619 02:43:30.414215 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:30.414222 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:30.414277 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:30.414411 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:30.415280 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:30.416641 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:30.417765 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:30.418342 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:30.418439 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:30.421323 (+ 2884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:30.421328 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:30.421328 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:30.421340 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:30.423321 (+ 1981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:30.423725 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:30.423728 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:30.423728 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:30.423853 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:30.424060 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:30.424061 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:30.424061 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:30.424343 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:30.424497 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:30.424941 (+ 444us) server_negotiation.cc:299] Negotiation successful
0619 02:43:30.425081 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":92,"threads_started":1}
I20260619 02:43:30.425354 23134 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:43:30.432901 23134 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f4d4c49ba53e4e67b4d83a85f89e7cba: Generated new TSK 0
I20260619 02:43:30.433082 23134 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.241.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab.
Entry for principal kudu/127.5.241.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab.
I20260619 02:43:30.458225 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.65:0
--local_ip_for_outbound_sockets=127.5.241.65
--webserver_interface=127.5.241.65
--webserver_port=0
--tserver_master_addrs=127.5.241.126:40231
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab
--principal=kudu/127.5.241.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:36635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:43:30.538940 23149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:30.539147 23149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:30.539171 23149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:30.540350 23149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:30.540393 23149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:30.540448 23149 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.65
I20260619 02:43:30.541961 23149 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:36635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab
--principal=kudu/127.5.241.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.241.65
--webserver_port=0
--tserver_master_addrs=127.5.241.126:40231
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23149
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.65
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:43:30.542217 23149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:30.542474 23149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:30.547377 23154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:30.547370 23157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:30.547371 23155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:30.547536 23149 server_base.cc:1034] running on GCE node
I20260619 02:43:30.547835 23149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:30.548028 23149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:30.549160 23149 hybrid_clock.cc:648] HybridClock initialized: now 1781837010549140 us; error 37 us; skew 500 ppm
Jun 19 02:43:30 dist-test-slave-zpbg krb5kdc[23042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837010, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:30.551713 23149 init.cc:377] Logged in from keytab as kudu/127.5.241.65@KRBTEST.COM (short username kudu)
I20260619 02:43:30.552170 23149 webserver.cc:460] Webserver started at http://127.5.241.65:44997/ using document root <none> and password file <none>
I20260619 02:43:30.552382 23149 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:30.552424 23149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:30.552510 23149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:43:30.553601 23149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/ts-0/data/instance:
uuid: "df17d8367f794c0290ba6bce5c21f435"
format_stamp: "Formatted at 2026-06-19 02:43:30 on dist-test-slave-zpbg"
server_key: "9c75b7402df2ee28071fbf083e12a12d"
server_key_iv: "03835b4827f99289b3b71184ecb405c8"
server_key_version: "encryptionkey@0"
I20260619 02:43:30.553903 23149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/ts-0/wal/instance:
uuid: "df17d8367f794c0290ba6bce5c21f435"
format_stamp: "Formatted at 2026-06-19 02:43:30 on dist-test-slave-zpbg"
server_key: "9c75b7402df2ee28071fbf083e12a12d"
server_key_iv: "03835b4827f99289b3b71184ecb405c8"
server_key_version: "encryptionkey@0"
I20260619 02:43:30.555224 23149 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:43:30.555918 23164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:30.556098 23149 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:43:30.556185 23149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/ts-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/ts-0/wal
uuid: "df17d8367f794c0290ba6bce5c21f435"
format_stamp: "Formatted at 2026-06-19 02:43:30 on dist-test-slave-zpbg"
server_key: "9c75b7402df2ee28071fbf083e12a12d"
server_key_iv: "03835b4827f99289b3b71184ecb405c8"
server_key_version: "encryptionkey@0"
I20260619 02:43:30.556243 23149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:30.574077 23149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:30.576431 23149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:30.576536 23149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:30.576720 23149 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:43:30.576962 23149 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:43:30.576990 23149 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:30.577013 23149 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:43:30.577024 23149 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:30.582619 23149 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.65:37883
I20260619 02:43:30.582679 23277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.65:37883 every 8 connection(s)
I20260619 02:43:30.583050 23149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
Jun 19 02:43:30 dist-test-slave-zpbg krb5kdc[23042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837010, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.65@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:43:30.592437 23145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:30.583793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:34827 (local address 127.5.241.126:40231)
0619 02:43:30.583850 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:30.583852 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:30.584069 (+ 217us) server_negotiation.cc:407] Connection header received
0619 02:43:30.584600 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:30.584601 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:30.584617 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:30.584681 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:30.585653 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:30.586170 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:30.587245 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:30.587464 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:30.587553 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:30.589711 (+ 2158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:30.589713 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:30.589713 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:30.589721 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:30.591038 (+ 1317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:30.591458 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:30.591459 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:30.591460 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:30.591520 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:30.591787 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:30.591788 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:30.591788 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:30.591903 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:30.591976 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:30.592258 (+ 282us) server_negotiation.cc:299] Negotiation successful
0619 02:43:30.592346 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 02:43:30.592610 23280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:30.583766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:40231 (local address 127.5.241.65:34827)
0619 02:43:30.583954 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:30.583988 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:30.584494 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:30.584778 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:30.584782 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:30.585342 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:43:30.585551 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:30.585555 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:30.586290 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:30.586291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:30.587151 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:30.587153 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:30.587563 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:30.587564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:30.587686 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:30.588216 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:43:30.588224 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:43:30.589614 (+ 1390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:43:30.591130 (+ 1516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:30.591137 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:30.591140 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:30.591367 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:30.591589 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:30.591590 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:30.591591 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:30.591701 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:30.591982 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:43:30.591988 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:43:30.592122 (+ 134us) client_negotiation.cc:769] Sending connection context
0619 02:43:30.592229 (+ 107us) client_negotiation.cc:240] Negotiation successful
0619 02:43:30.592382 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":98,"threads_started":1}
I20260619 02:43:30.593078 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 23149
I20260619 02:43:30.593122 23278 heartbeater.cc:347] Connected to a master server at 127.5.241.126:40231
I20260619 02:43:30.593194 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/ts-0/wal/instance
I20260619 02:43:30.593214 23278 heartbeater.cc:464] Registering TS with master...
I20260619 02:43:30.593410 23278 heartbeater.cc:511] Master 127.5.241.126:40231 requested a full tablet report, sending...
I20260619 02:43:30.593415 6085 external_mini_cluster.cc:1428] Setting key b65f9d6a07d8c4022d35952214388b07
I20260619 02:43:30.594090 23086 ts_manager.cc:194] Registered new tserver with Master: df17d8367f794c0290ba6bce5c21f435 (127.5.241.65:37883)
I20260619 02:43:30.595016 23086 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.65@KRBTEST.COM'} at 127.5.241.65:34827
WARNING: no policy specified for kudu/127.5.241.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab.
Entry for principal kudu/127.5.241.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab.
I20260619 02:43:30.617564 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.66:0
--local_ip_for_outbound_sockets=127.5.241.66
--webserver_interface=127.5.241.66
--webserver_port=0
--tserver_master_addrs=127.5.241.126:40231
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab
--principal=kudu/127.5.241.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:36635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:43:30.696354 23283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:30.696553 23283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:30.696576 23283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:30.697713 23283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:30.697755 23283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:30.697781 23283 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.66
I20260619 02:43:30.699220 23283 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:36635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab
--principal=kudu/127.5.241.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.241.66
--webserver_port=0
--tserver_master_addrs=127.5.241.126:40231
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23283
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.66
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:43:30.699456 23283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:30.699697 23283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:30.704653 23291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:30.704842 23289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:30.704861 23288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:30.704998 23283 server_base.cc:1034] running on GCE node
I20260619 02:43:30.705209 23283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:30.705474 23283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:30.706661 23283 hybrid_clock.cc:648] HybridClock initialized: now 1781837010706650 us; error 41 us; skew 500 ppm
Jun 19 02:43:30 dist-test-slave-zpbg krb5kdc[23042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837010, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:30.709241 23283 init.cc:377] Logged in from keytab as kudu/127.5.241.66@KRBTEST.COM (short username kudu)
I20260619 02:43:30.709719 23283 webserver.cc:460] Webserver started at http://127.5.241.66:38253/ using document root <none> and password file <none>
I20260619 02:43:30.709964 23283 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:30.710033 23283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:30.710184 23283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:43:30.711324 23283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/ts-1/data/instance:
uuid: "e05ddbd262bb4e6d9a4379fb9818346c"
format_stamp: "Formatted at 2026-06-19 02:43:30 on dist-test-slave-zpbg"
server_key: "b188c5593c7a10a9657a8ca2f7ee2e41"
server_key_iv: "335a721089c20c66ec8b4ab13fda71f5"
server_key_version: "encryptionkey@0"
I20260619 02:43:30.711680 23283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/ts-1/wal/instance:
uuid: "e05ddbd262bb4e6d9a4379fb9818346c"
format_stamp: "Formatted at 2026-06-19 02:43:30 on dist-test-slave-zpbg"
server_key: "b188c5593c7a10a9657a8ca2f7ee2e41"
server_key_iv: "335a721089c20c66ec8b4ab13fda71f5"
server_key_version: "encryptionkey@0"
I20260619 02:43:30.713097 23283 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:43:30.713829 23298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:30.714003 23283 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:43:30.714098 23283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/ts-1/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/ts-1/wal
uuid: "e05ddbd262bb4e6d9a4379fb9818346c"
format_stamp: "Formatted at 2026-06-19 02:43:30 on dist-test-slave-zpbg"
server_key: "b188c5593c7a10a9657a8ca2f7ee2e41"
server_key_iv: "335a721089c20c66ec8b4ab13fda71f5"
server_key_version: "encryptionkey@0"
I20260619 02:43:30.714184 23283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:30.728005 23283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:30.729820 23283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:30.729983 23283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:30.730216 23283 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:43:30.730522 23283 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:43:30.730580 23283 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:30.730623 23283 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:43:30.730651 23283 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:30.736557 23283 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.66:46325
I20260619 02:43:30.736605 23411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.66:46325 every 8 connection(s)
I20260619 02:43:30.736904 23283 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
I20260619 02:43:30.741953 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 23283
I20260619 02:43:30.742069 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/ts-1/wal/instance
I20260619 02:43:30.742290 6085 external_mini_cluster.cc:1428] Setting key 9ba2ef7316503a834f50a688ddc4046b
Jun 19 02:43:30 dist-test-slave-zpbg krb5kdc[23042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837010, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.66@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:43:30.746372 23414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:30.737545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:40231 (local address 127.5.241.66:43943)
0619 02:43:30.737743 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:30.737779 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:30.738251 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:30.738466 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:30.738469 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:30.739084 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:43:30.739281 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:30.739287 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:30.739921 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:30.739922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:30.740705 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:30.740707 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:30.741033 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:30.741034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:30.741127 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:30.741564 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:43:30.741571 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:43:30.743139 (+ 1568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:43:30.744941 (+ 1802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:30.744948 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:30.744951 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:30.745284 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:30.745476 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:30.745477 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:30.745478 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:30.745609 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:30.745831 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:43:30.745836 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:43:30.745975 (+ 139us) client_negotiation.cc:769] Sending connection context
0619 02:43:30.746094 (+ 119us) client_negotiation.cc:240] Negotiation successful
0619 02:43:30.746218 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":112,"threads_started":1}
I20260619 02:43:30.746376 23145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:30.737570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:43943 (local address 127.5.241.126:40231)
0619 02:43:30.737643 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:30.737646 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:30.737868 (+ 222us) server_negotiation.cc:407] Connection header received
0619 02:43:30.738348 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:30.738349 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:30.738359 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:30.738413 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:30.739361 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:30.739842 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:30.740784 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:30.740962 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:30.741013 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:30.743245 (+ 2232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:30.743246 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:30.743246 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:30.743254 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:30.744852 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:30.745364 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:30.745365 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:30.745365 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:30.745414 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:30.745670 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:30.745671 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:30.745671 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:30.745768 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:30.745816 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:30.746136 (+ 320us) server_negotiation.cc:299] Negotiation successful
0619 02:43:30.746223 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 02:43:30.746853 23412 heartbeater.cc:347] Connected to a master server at 127.5.241.126:40231
I20260619 02:43:30.746975 23412 heartbeater.cc:464] Registering TS with master...
I20260619 02:43:30.747164 23412 heartbeater.cc:511] Master 127.5.241.126:40231 requested a full tablet report, sending...
I20260619 02:43:30.747653 23086 ts_manager.cc:194] Registered new tserver with Master: e05ddbd262bb4e6d9a4379fb9818346c (127.5.241.66:46325)
I20260619 02:43:30.748150 23086 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.66@KRBTEST.COM'} at 127.5.241.66:43943
WARNING: no policy specified for kudu/127.5.241.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab.
Entry for principal kudu/127.5.241.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab.
I20260619 02:43:30.769284 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.67:0
--local_ip_for_outbound_sockets=127.5.241.67
--webserver_interface=127.5.241.67
--webserver_port=0
--tserver_master_addrs=127.5.241.126:40231
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab
--principal=kudu/127.5.241.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:36635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:43:30.848527 23417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:30.848724 23417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:30.848747 23417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:30.849799 23417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:30.849831 23417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:30.849856 23417 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.67
I20260619 02:43:30.851258 23417 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:36635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab
--principal=kudu/127.5.241.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.241.67
--webserver_port=0
--tserver_master_addrs=127.5.241.126:40231
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23417
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.67
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:43:30.851434 23417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:30.851594 23417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:30.856288 23423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:30.856277 23422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:30.856273 23425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:30.856729 23417 server_base.cc:1034] running on GCE node
I20260619 02:43:30.856884 23417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:30.857066 23417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:30.858192 23417 hybrid_clock.cc:648] HybridClock initialized: now 1781837010858183 us; error 26 us; skew 500 ppm
Jun 19 02:43:30 dist-test-slave-zpbg krb5kdc[23042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837010, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:30.860677 23417 init.cc:377] Logged in from keytab as kudu/127.5.241.67@KRBTEST.COM (short username kudu)
I20260619 02:43:30.861075 23417 webserver.cc:460] Webserver started at http://127.5.241.67:38005/ using document root <none> and password file <none>
I20260619 02:43:30.861316 23417 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:30.861385 23417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:30.861529 23417 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:43:30.862537 23417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/ts-2/data/instance:
uuid: "639ff99251f5431c95a647cafe002de1"
format_stamp: "Formatted at 2026-06-19 02:43:30 on dist-test-slave-zpbg"
server_key: "bcd0e86d6cadccc14bb05f446a603298"
server_key_iv: "0eccb645422f07dd5b004a19fcd30a36"
server_key_version: "encryptionkey@0"
I20260619 02:43:30.863018 23417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/ts-2/wal/instance:
uuid: "639ff99251f5431c95a647cafe002de1"
format_stamp: "Formatted at 2026-06-19 02:43:30 on dist-test-slave-zpbg"
server_key: "bcd0e86d6cadccc14bb05f446a603298"
server_key_iv: "0eccb645422f07dd5b004a19fcd30a36"
server_key_version: "encryptionkey@0"
I20260619 02:43:30.864341 23417 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:43:30.865017 23432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:30.865170 23417 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:30.865271 23417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/ts-2/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/ts-2/wal
uuid: "639ff99251f5431c95a647cafe002de1"
format_stamp: "Formatted at 2026-06-19 02:43:30 on dist-test-slave-zpbg"
server_key: "bcd0e86d6cadccc14bb05f446a603298"
server_key_iv: "0eccb645422f07dd5b004a19fcd30a36"
server_key_version: "encryptionkey@0"
I20260619 02:43:30.865352 23417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:30.894438 23417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:30.896433 23417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:30.896608 23417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:30.896859 23417 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:43:30.897189 23417 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:43:30.897245 23417 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:30.897320 23417 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:43:30.897351 23417 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:30.903667 23417 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.67:34709
I20260619 02:43:30.903712 23545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.67:34709 every 8 connection(s)
I20260619 02:43:30.904045 23417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
Jun 19 02:43:30 dist-test-slave-zpbg krb5kdc[23042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837010, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.67@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:43:30.913089 23145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:30.904881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:36555 (local address 127.5.241.126:40231)
0619 02:43:30.904954 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:30.904957 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:30.905122 (+ 165us) server_negotiation.cc:407] Connection header received
0619 02:43:30.905694 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:30.905695 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:30.905705 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:30.905762 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:30.906667 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:30.907331 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:30.908266 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:30.908445 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:30.908496 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:30.910434 (+ 1938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:30.910435 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:30.910435 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:30.910442 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:30.911814 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:30.912240 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:30.912241 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:30.912241 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:30.912337 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:30.912573 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:30.912574 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:30.912574 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:30.912684 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:30.912750 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:30.912969 (+ 219us) server_negotiation.cc:299] Negotiation successful
0619 02:43:30.913027 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 02:43:30.913286 23548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:30.904724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:40231 (local address 127.5.241.67:36555)
0619 02:43:30.904998 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:30.905037 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:30.905534 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:30.905834 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:30.905837 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:30.906386 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:43:30.906572 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:30.906576 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:30.907410 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:30.907410 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:30.908202 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:30.908204 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:30.908515 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:30.908516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:30.908608 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:30.909025 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:43:30.909033 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:43:30.910354 (+ 1321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:43:30.911889 (+ 1535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:30.911895 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:30.911897 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:30.912167 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:30.912406 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:30.912407 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:30.912407 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:30.912518 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:30.912755 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:43:30.912759 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:43:30.912863 (+ 104us) client_negotiation.cc:769] Sending connection context
0619 02:43:30.912942 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 02:43:30.913052 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":195,"threads_started":1}
I20260619 02:43:30.913754 23546 heartbeater.cc:347] Connected to a master server at 127.5.241.126:40231
I20260619 02:43:30.913863 23546 heartbeater.cc:464] Registering TS with master...
I20260619 02:43:30.914036 23546 heartbeater.cc:511] Master 127.5.241.126:40231 requested a full tablet report, sending...
I20260619 02:43:30.914112 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 23417
I20260619 02:43:30.914222 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/ts-2/wal/instance
I20260619 02:43:30.914407 23086 ts_manager.cc:194] Registered new tserver with Master: 639ff99251f5431c95a647cafe002de1 (127.5.241.67:34709)
I20260619 02:43:30.914456 6085 external_mini_cluster.cc:1428] Setting key 96fac2474687e6eb619a756e404a18b2
I20260619 02:43:30.914939 23086 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.67@KRBTEST.COM'} at 127.5.241.67:36555
I20260619 02:43:30.915308 6085 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:43:30.921938 23145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:30.916610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58618 (local address 127.5.241.126:40231)
0619 02:43:30.916687 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:30.916689 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:30.916786 (+ 97us) server_negotiation.cc:407] Connection header received
0619 02:43:30.916863 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:30.916864 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:30.916872 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:30.916910 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:30.917410 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:30.917924 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:30.918589 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:30.918802 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:30.918861 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:30.919967 (+ 1106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:30.919969 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:30.919969 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:30.919974 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:30.921065 (+ 1091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:30.921354 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:30.921354 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:30.921355 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:30.921400 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:30.921517 (+ 117us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:30.921517 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:30.921517 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:30.921630 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:30.921697 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:30.921804 (+ 107us) server_negotiation.cc:299] Negotiation successful
0619 02:43:30.921856 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 02:43:30.923537 23087 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58618:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:43:30.925096 23087 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:43:30.931609 23556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:30.927457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:37883 (local address 127.0.0.1:33596)
0619 02:43:30.927901 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:30.927910 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:30.927951 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:30.928284 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:30.928285 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:30.928399 (+ 114us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:43:30.928454 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:30.928457 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:30.929306 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:30.929307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:30.930520 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:30.930522 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:30.931304 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:30.931305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:30.931392 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:30.931406 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 02:43:30.931473 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 02:43:30.931527 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":62,"threads_started":1}
I20260619 02:43:30.931700 23558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:30.927735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53206 (local address 127.5.241.66:46325)
0619 02:43:30.927902 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:30.927906 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:30.927917 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:43:30.927961 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:30.927963 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:30.927980 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:30.928270 (+ 290us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:43:30.928354 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:30.929507 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:30.930621 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:30.931359 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:30.931442 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:30.931497 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:30.931575 (+ 78us) server_negotiation.cc:299] Negotiation successful
0619 02:43:30.931620 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":97,"threads_started":1}
I20260619 02:43:30.931692 23555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:30.927420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.66:46325 (local address 127.0.0.1:53206)
0619 02:43:30.927588 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:30.927640 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:30.927705 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:30.928089 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:30.928092 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:30.928094 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:43:30.928181 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:30.928182 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:30.929642 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:30.929643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:30.930525 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:30.930527 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:30.931442 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:30.931443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:30.931509 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:30.931512 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:43:30.931562 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 02:43:30.931594 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":55,"threads_started":1}
I20260619 02:43:30.932027 23557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:30.927575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33596 (local address 127.5.241.65:37883)
0619 02:43:30.928090 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:30.928093 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:30.928103 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:43:30.928159 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:30.928161 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:30.928176 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:30.928281 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:43:30.928511 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:30.929182 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:30.930613 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:30.931199 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:30.931301 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:30.931364 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:30.931880 (+ 516us) server_negotiation.cc:299] Negotiation successful
0619 02:43:30.931931 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":84,"threads_started":1}
I20260619 02:43:30.932220 23554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:30.927615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.67:34709 (local address 127.0.0.1:38400)
0619 02:43:30.927920 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:30.927931 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:30.927999 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:30.928313 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:30.928314 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:30.928316 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:43:30.928356 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:30.928357 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:30.929616 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:30.929617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:30.930682 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:30.930684 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:30.931617 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:30.931618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:30.931680 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:30.931684 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:43:30.932091 (+ 407us) client_negotiation.cc:240] Negotiation successful
0619 02:43:30.932133 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"mutex_wait_us":3,"thread_start_us":36,"threads_started":1}
I20260619 02:43:30.932725 23212 tablet_service.cc:1467] Processing CreateTablet for tablet 261235aa8c2b4dab95d7df22543c0576 (DEFAULT_TABLE table=test-table [id=3299ab9ac3754a639e6a6d56d26dfa3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:43:30.932701 23346 tablet_service.cc:1467] Processing CreateTablet for tablet 261235aa8c2b4dab95d7df22543c0576 (DEFAULT_TABLE table=test-table [id=3299ab9ac3754a639e6a6d56d26dfa3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:43:30.932986 23212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 261235aa8c2b4dab95d7df22543c0576. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:30.932986 23346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 261235aa8c2b4dab95d7df22543c0576. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:30.935005 23559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:30.927722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38400 (local address 127.5.241.67:34709)
0619 02:43:30.927886 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:30.927890 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:30.928009 (+ 119us) server_negotiation.cc:407] Connection header received
0619 02:43:30.928061 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:30.928064 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:30.928080 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:30.928160 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:43:30.928441 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:30.929519 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:30.930757 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:30.931473 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:30.931534 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:30.931583 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:30.934886 (+ 3303us) server_negotiation.cc:299] Negotiation successful
0619 02:43:30.934935 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":97,"threads_started":1}
I20260619 02:43:30.935456 23560 tablet_bootstrap.cc:492] T 261235aa8c2b4dab95d7df22543c0576 P e05ddbd262bb4e6d9a4379fb9818346c: Bootstrap starting.
I20260619 02:43:30.935693 23480 tablet_service.cc:1467] Processing CreateTablet for tablet 261235aa8c2b4dab95d7df22543c0576 (DEFAULT_TABLE table=test-table [id=3299ab9ac3754a639e6a6d56d26dfa3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:43:30.935937 23480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 261235aa8c2b4dab95d7df22543c0576. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:30.936110 23560 tablet_bootstrap.cc:654] T 261235aa8c2b4dab95d7df22543c0576 P e05ddbd262bb4e6d9a4379fb9818346c: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:30.936523 23560 log.cc:824] T 261235aa8c2b4dab95d7df22543c0576 P e05ddbd262bb4e6d9a4379fb9818346c: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:30.937163 23560 tablet_bootstrap.cc:492] T 261235aa8c2b4dab95d7df22543c0576 P e05ddbd262bb4e6d9a4379fb9818346c: No bootstrap required, opened a new log
I20260619 02:43:30.937206 23560 ts_tablet_manager.cc:1397] T 261235aa8c2b4dab95d7df22543c0576 P e05ddbd262bb4e6d9a4379fb9818346c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:43:30.938308 23563 tablet_bootstrap.cc:492] T 261235aa8c2b4dab95d7df22543c0576 P 639ff99251f5431c95a647cafe002de1: Bootstrap starting.
I20260619 02:43:30.938655 23561 tablet_bootstrap.cc:492] T 261235aa8c2b4dab95d7df22543c0576 P df17d8367f794c0290ba6bce5c21f435: Bootstrap starting.
I20260619 02:43:30.938700 23560 raft_consensus.cc:348] T 261235aa8c2b4dab95d7df22543c0576 P e05ddbd262bb4e6d9a4379fb9818346c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e05ddbd262bb4e6d9a4379fb9818346c" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 46325 } } peers { permanent_uuid: "639ff99251f5431c95a647cafe002de1" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 34709 } } peers { permanent_uuid: "df17d8367f794c0290ba6bce5c21f435" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 37883 } }
I20260619 02:43:30.938920 23560 raft_consensus.cc:374] T 261235aa8c2b4dab95d7df22543c0576 P e05ddbd262bb4e6d9a4379fb9818346c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:30.938961 23563 tablet_bootstrap.cc:654] T 261235aa8c2b4dab95d7df22543c0576 P 639ff99251f5431c95a647cafe002de1: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:30.938971 23560 raft_consensus.cc:729] T 261235aa8c2b4dab95d7df22543c0576 P e05ddbd262bb4e6d9a4379fb9818346c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e05ddbd262bb4e6d9a4379fb9818346c, State: Initialized, Role: FOLLOWER
I20260619 02:43:30.939095 23560 consensus_queue.cc:260] T 261235aa8c2b4dab95d7df22543c0576 P e05ddbd262bb4e6d9a4379fb9818346c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e05ddbd262bb4e6d9a4379fb9818346c" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 46325 } } peers { permanent_uuid: "639ff99251f5431c95a647cafe002de1" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 34709 } } peers { permanent_uuid: "df17d8367f794c0290ba6bce5c21f435" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 37883 } }
I20260619 02:43:30.939302 23561 tablet_bootstrap.cc:654] T 261235aa8c2b4dab95d7df22543c0576 P df17d8367f794c0290ba6bce5c21f435: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:30.939431 23560 ts_tablet_manager.cc:1428] T 261235aa8c2b4dab95d7df22543c0576 P e05ddbd262bb4e6d9a4379fb9818346c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:43:30.939460 23412 heartbeater.cc:503] Master 127.5.241.126:40231 was elected leader, sending a full tablet report...
I20260619 02:43:30.939561 23563 log.cc:824] T 261235aa8c2b4dab95d7df22543c0576 P 639ff99251f5431c95a647cafe002de1: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:30.940169 23563 tablet_bootstrap.cc:492] T 261235aa8c2b4dab95d7df22543c0576 P 639ff99251f5431c95a647cafe002de1: No bootstrap required, opened a new log
I20260619 02:43:30.940250 23563 ts_tablet_manager.cc:1397] T 261235aa8c2b4dab95d7df22543c0576 P 639ff99251f5431c95a647cafe002de1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:43:30.940416 23561 log.cc:824] T 261235aa8c2b4dab95d7df22543c0576 P df17d8367f794c0290ba6bce5c21f435: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:30.941630 23563 raft_consensus.cc:348] T 261235aa8c2b4dab95d7df22543c0576 P 639ff99251f5431c95a647cafe002de1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e05ddbd262bb4e6d9a4379fb9818346c" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 46325 } } peers { permanent_uuid: "639ff99251f5431c95a647cafe002de1" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 34709 } } peers { permanent_uuid: "df17d8367f794c0290ba6bce5c21f435" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 37883 } }
I20260619 02:43:30.941758 23563 raft_consensus.cc:374] T 261235aa8c2b4dab95d7df22543c0576 P 639ff99251f5431c95a647cafe002de1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:30.941780 23563 raft_consensus.cc:729] T 261235aa8c2b4dab95d7df22543c0576 P 639ff99251f5431c95a647cafe002de1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ff99251f5431c95a647cafe002de1, State: Initialized, Role: FOLLOWER
I20260619 02:43:30.941915 23563 consensus_queue.cc:260] T 261235aa8c2b4dab95d7df22543c0576 P 639ff99251f5431c95a647cafe002de1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e05ddbd262bb4e6d9a4379fb9818346c" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 46325 } } peers { permanent_uuid: "639ff99251f5431c95a647cafe002de1" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 34709 } } peers { permanent_uuid: "df17d8367f794c0290ba6bce5c21f435" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 37883 } }
I20260619 02:43:30.942186 23563 ts_tablet_manager.cc:1428] T 261235aa8c2b4dab95d7df22543c0576 P 639ff99251f5431c95a647cafe002de1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:43:30.942201 23546 heartbeater.cc:503] Master 127.5.241.126:40231 was elected leader, sending a full tablet report...
I20260619 02:43:30.942770 23561 tablet_bootstrap.cc:492] T 261235aa8c2b4dab95d7df22543c0576 P df17d8367f794c0290ba6bce5c21f435: No bootstrap required, opened a new log
I20260619 02:43:30.942857 23561 ts_tablet_manager.cc:1397] T 261235aa8c2b4dab95d7df22543c0576 P df17d8367f794c0290ba6bce5c21f435: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260619 02:43:30.944273 23561 raft_consensus.cc:348] T 261235aa8c2b4dab95d7df22543c0576 P df17d8367f794c0290ba6bce5c21f435 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e05ddbd262bb4e6d9a4379fb9818346c" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 46325 } } peers { permanent_uuid: "639ff99251f5431c95a647cafe002de1" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 34709 } } peers { permanent_uuid: "df17d8367f794c0290ba6bce5c21f435" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 37883 } }
I20260619 02:43:30.944442 23561 raft_consensus.cc:374] T 261235aa8c2b4dab95d7df22543c0576 P df17d8367f794c0290ba6bce5c21f435 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:30.944486 23561 raft_consensus.cc:729] T 261235aa8c2b4dab95d7df22543c0576 P df17d8367f794c0290ba6bce5c21f435 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df17d8367f794c0290ba6bce5c21f435, State: Initialized, Role: FOLLOWER
I20260619 02:43:30.944595 23561 consensus_queue.cc:260] T 261235aa8c2b4dab95d7df22543c0576 P df17d8367f794c0290ba6bce5c21f435 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e05ddbd262bb4e6d9a4379fb9818346c" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 46325 } } peers { permanent_uuid: "639ff99251f5431c95a647cafe002de1" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 34709 } } peers { permanent_uuid: "df17d8367f794c0290ba6bce5c21f435" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 37883 } }
I20260619 02:43:30.944834 23561 ts_tablet_manager.cc:1428] T 261235aa8c2b4dab95d7df22543c0576 P df17d8367f794c0290ba6bce5c21f435: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 02:43:30.944900 23278 heartbeater.cc:503] Master 127.5.241.126:40231 was elected leader, sending a full tablet report...
W20260619 02:43:30.987447 23413 tablet.cc:2369] T 261235aa8c2b4dab95d7df22543c0576 P e05ddbd262bb4e6d9a4379fb9818346c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:43:31.083853 23279 tablet.cc:2369] T 261235aa8c2b4dab95d7df22543c0576 P df17d8367f794c0290ba6bce5c21f435: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:43:31.154633 23547 tablet.cc:2369] T 261235aa8c2b4dab95d7df22543c0576 P 639ff99251f5431c95a647cafe002de1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:43:31.286732 23568 raft_consensus.cc:482] T 261235aa8c2b4dab95d7df22543c0576 P df17d8367f794c0290ba6bce5c21f435 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:43:31.286974 23568 raft_consensus.cc:504] T 261235aa8c2b4dab95d7df22543c0576 P df17d8367f794c0290ba6bce5c21f435 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e05ddbd262bb4e6d9a4379fb9818346c" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 46325 } } peers { permanent_uuid: "639ff99251f5431c95a647cafe002de1" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 34709 } } peers { permanent_uuid: "df17d8367f794c0290ba6bce5c21f435" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 37883 } }
I20260619 02:43:31.287425 23568 leader_election.cc:288] T 261235aa8c2b4dab95d7df22543c0576 P df17d8367f794c0290ba6bce5c21f435 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e05ddbd262bb4e6d9a4379fb9818346c (127.5.241.66:46325), 639ff99251f5431c95a647cafe002de1 (127.5.241.67:34709)
I20260619 02:43:31.290807 23558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:31.287736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:35217 (local address 127.5.241.66:46325)
0619 02:43:31.287780 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:31.287783 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:31.287853 (+ 70us) server_negotiation.cc:407] Connection header received
0619 02:43:31.288002 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:31.288004 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:31.288019 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:31.288092 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:43:31.288296 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:31.288861 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:31.290006 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:31.290424 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:31.290504 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:31.290515 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:31.290672 (+ 157us) server_negotiation.cc:299] Negotiation successful
0619 02:43:31.290711 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 02:43:31.290820 23569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:31.287658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.66:46325 (local address 127.5.241.65:35217)
0619 02:43:31.287780 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:31.287792 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:31.287857 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:31.288125 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:31.288126 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:31.288128 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:43:31.288215 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:31.288217 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:31.288965 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:31.288966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:31.289902 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:31.289904 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:31.290518 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:31.290519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:31.290595 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:31.290599 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:43:31.290666 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 02:43:31.290717 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"mutex_wait_us":46,"thread_start_us":74,"threads_started":1}
I20260619 02:43:31.290884 23559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:31.287942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:59487 (local address 127.5.241.67:34709)
0619 02:43:31.288019 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:31.288021 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:31.288029 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:43:31.288137 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:31.288138 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:31.288148 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:31.288210 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:43:31.288412 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:31.288870 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:31.290056 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:31.290510 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:31.290597 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:31.290605 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:31.290753 (+ 148us) server_negotiation.cc:299] Negotiation successful
0619 02:43:31.290813 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 02:43:31.290928 23570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:31.287849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.67:34709 (local address 127.5.241.65:59487)
0619 02:43:31.287959 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:31.287970 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:31.288038 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:31.288223 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:31.288224 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:31.288226 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:43:31.288317 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:31.288319 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:31.288977 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:31.288978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:31.289948 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:31.289950 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:31.290597 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:31.290598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:31.290667 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:31.290671 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:43:31.290747 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 02:43:31.290809 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"mutex_wait_us":1,"thread_start_us":50,"threads_started":1}
I20260619 02:43:31.291254 23366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "261235aa8c2b4dab95d7df22543c0576" candidate_uuid: "df17d8367f794c0290ba6bce5c21f435" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e05ddbd262bb4e6d9a4379fb9818346c" is_pre_election: true
I20260619 02:43:31.291281 23500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "261235aa8c2b4dab95d7df22543c0576" candidate_uuid: "df17d8367f794c0290ba6bce5c21f435" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ff99251f5431c95a647cafe002de1" is_pre_election: true
I20260619 02:43:31.291371 23366 raft_consensus.cc:2454] T 261235aa8c2b4dab95d7df22543c0576 P e05ddbd262bb4e6d9a4379fb9818346c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df17d8367f794c0290ba6bce5c21f435 in term 0.
I20260619 02:43:31.291441 23500 raft_consensus.cc:2454] T 261235aa8c2b4dab95d7df22543c0576 P 639ff99251f5431c95a647cafe002de1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df17d8367f794c0290ba6bce5c21f435 in term 0.
I20260619 02:43:31.291640 23165 leader_election.cc:302] T 261235aa8c2b4dab95d7df22543c0576 P df17d8367f794c0290ba6bce5c21f435 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 639ff99251f5431c95a647cafe002de1, df17d8367f794c0290ba6bce5c21f435; no voters:
I20260619 02:43:31.291790 23568 raft_consensus.cc:2781] T 261235aa8c2b4dab95d7df22543c0576 P df17d8367f794c0290ba6bce5c21f435 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:43:31.291875 23568 raft_consensus.cc:482] T 261235aa8c2b4dab95d7df22543c0576 P df17d8367f794c0290ba6bce5c21f435 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:43:31.291909 23568 raft_consensus.cc:3037] T 261235aa8c2b4dab95d7df22543c0576 P df17d8367f794c0290ba6bce5c21f435 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:31.292623 23568 raft_consensus.cc:504] T 261235aa8c2b4dab95d7df22543c0576 P df17d8367f794c0290ba6bce5c21f435 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e05ddbd262bb4e6d9a4379fb9818346c" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 46325 } } peers { permanent_uuid: "639ff99251f5431c95a647cafe002de1" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 34709 } } peers { permanent_uuid: "df17d8367f794c0290ba6bce5c21f435" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 37883 } }
I20260619 02:43:31.292773 23568 leader_election.cc:288] T 261235aa8c2b4dab95d7df22543c0576 P df17d8367f794c0290ba6bce5c21f435 [CANDIDATE]: Term 1 election: Requested vote from peers e05ddbd262bb4e6d9a4379fb9818346c (127.5.241.66:46325), 639ff99251f5431c95a647cafe002de1 (127.5.241.67:34709)
I20260619 02:43:31.292981 23366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "261235aa8c2b4dab95d7df22543c0576" candidate_uuid: "df17d8367f794c0290ba6bce5c21f435" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e05ddbd262bb4e6d9a4379fb9818346c"
I20260619 02:43:31.293012 23500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "261235aa8c2b4dab95d7df22543c0576" candidate_uuid: "df17d8367f794c0290ba6bce5c21f435" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ff99251f5431c95a647cafe002de1"
I20260619 02:43:31.293076 23366 raft_consensus.cc:3037] T 261235aa8c2b4dab95d7df22543c0576 P e05ddbd262bb4e6d9a4379fb9818346c [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:31.293107 23500 raft_consensus.cc:3037] T 261235aa8c2b4dab95d7df22543c0576 P 639ff99251f5431c95a647cafe002de1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:31.293771 23366 raft_consensus.cc:2454] T 261235aa8c2b4dab95d7df22543c0576 P e05ddbd262bb4e6d9a4379fb9818346c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df17d8367f794c0290ba6bce5c21f435 in term 1.
I20260619 02:43:31.293974 23165 leader_election.cc:302] T 261235aa8c2b4dab95d7df22543c0576 P df17d8367f794c0290ba6bce5c21f435 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: df17d8367f794c0290ba6bce5c21f435, e05ddbd262bb4e6d9a4379fb9818346c; no voters:
I20260619 02:43:31.293972 23500 raft_consensus.cc:2454] T 261235aa8c2b4dab95d7df22543c0576 P 639ff99251f5431c95a647cafe002de1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df17d8367f794c0290ba6bce5c21f435 in term 1.
I20260619 02:43:31.294106 23568 raft_consensus.cc:2781] T 261235aa8c2b4dab95d7df22543c0576 P df17d8367f794c0290ba6bce5c21f435 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:43:31.294282 23568 raft_consensus.cc:686] T 261235aa8c2b4dab95d7df22543c0576 P df17d8367f794c0290ba6bce5c21f435 [term 1 LEADER]: Becoming Leader. State: Replica: df17d8367f794c0290ba6bce5c21f435, State: Running, Role: LEADER
I20260619 02:43:31.294400 23568 consensus_queue.cc:237] T 261235aa8c2b4dab95d7df22543c0576 P df17d8367f794c0290ba6bce5c21f435 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e05ddbd262bb4e6d9a4379fb9818346c" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 46325 } } peers { permanent_uuid: "639ff99251f5431c95a647cafe002de1" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 34709 } } peers { permanent_uuid: "df17d8367f794c0290ba6bce5c21f435" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 37883 } }
I20260619 02:43:31.295246 23087 catalog_manager.cc:5495] T 261235aa8c2b4dab95d7df22543c0576 P df17d8367f794c0290ba6bce5c21f435 reported cstate change: term changed from 0 to 1, leader changed from <none> to df17d8367f794c0290ba6bce5c21f435 (127.5.241.65). New cstate: current_term: 1 leader_uuid: "df17d8367f794c0290ba6bce5c21f435" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e05ddbd262bb4e6d9a4379fb9818346c" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 46325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ff99251f5431c95a647cafe002de1" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 34709 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df17d8367f794c0290ba6bce5c21f435" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 37883 } health_report { overall_health: HEALTHY } } }
I20260619 02:43:31.373834 23557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:31.371276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33606 (local address 127.5.241.65:37883)
0619 02:43:31.371347 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:31.371349 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:31.371360 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:43:31.371422 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:31.371422 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:31.371442 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:31.371519 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:43:31.371659 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:31.372193 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:31.373008 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:31.373209 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:31.373316 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:31.373484 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:43:31.373590 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:43:31.373684 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:31.373724 (+ 40us) server_negotiation.cc:299] Negotiation successful
0619 02:43:31.373747 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 02:43:31.375659 23366 raft_consensus.cc:1261] T 261235aa8c2b4dab95d7df22543c0576 P e05ddbd262bb4e6d9a4379fb9818346c [term 1 FOLLOWER]: Refusing update from remote peer df17d8367f794c0290ba6bce5c21f435: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:43:31.375679 23500 raft_consensus.cc:1261] T 261235aa8c2b4dab95d7df22543c0576 P 639ff99251f5431c95a647cafe002de1 [term 1 FOLLOWER]: Refusing update from remote peer df17d8367f794c0290ba6bce5c21f435: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:43:31.376044 23571 consensus_queue.cc:1035] T 261235aa8c2b4dab95d7df22543c0576 P df17d8367f794c0290ba6bce5c21f435 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e05ddbd262bb4e6d9a4379fb9818346c" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 46325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:43:31.376183 23568 consensus_queue.cc:1035] T 261235aa8c2b4dab95d7df22543c0576 P df17d8367f794c0290ba6bce5c21f435 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ff99251f5431c95a647cafe002de1" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 34709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:43:31.380486 23087 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58618:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:43:31.380589 23087 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58618:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:43:31.381275 23087 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f4d4c49ba53e4e67b4d83a85f89e7cba: Sending DeleteTablet for 3 replicas of tablet 261235aa8c2b4dab95d7df22543c0576
I20260619 02:43:31.381596 23212 tablet_service.cc:1514] Processing DeleteTablet for tablet 261235aa8c2b4dab95d7df22543c0576 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:43:31 UTC) from {username='kudu', principal='kudu/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:33596
I20260619 02:43:31.381654 23346 tablet_service.cc:1514] Processing DeleteTablet for tablet 261235aa8c2b4dab95d7df22543c0576 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:43:31 UTC) from {username='kudu', principal='kudu/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:53206
I20260619 02:43:31.381836 23480 tablet_service.cc:1514] Processing DeleteTablet for tablet 261235aa8c2b4dab95d7df22543c0576 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:43:31 UTC) from {username='kudu', principal='kudu/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:38400
I20260619 02:43:31.381868 23583 tablet_replica.cc:330] T 261235aa8c2b4dab95d7df22543c0576 P df17d8367f794c0290ba6bce5c21f435: stopping tablet replica
I20260619 02:43:31.381963 23583 raft_consensus.cc:2229] T 261235aa8c2b4dab95d7df22543c0576 P df17d8367f794c0290ba6bce5c21f435 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:43:31.382031 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 23149
I20260619 02:43:31.382041 23583 raft_consensus.cc:2258] T 261235aa8c2b4dab95d7df22543c0576 P df17d8367f794c0290ba6bce5c21f435 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:43:31.382148 23584 tablet_replica.cc:330] T 261235aa8c2b4dab95d7df22543c0576 P e05ddbd262bb4e6d9a4379fb9818346c: stopping tablet replica
I20260619 02:43:31.382236 23584 raft_consensus.cc:2229] T 261235aa8c2b4dab95d7df22543c0576 P e05ddbd262bb4e6d9a4379fb9818346c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:43:31.382362 23584 raft_consensus.cc:2258] T 261235aa8c2b4dab95d7df22543c0576 P e05ddbd262bb4e6d9a4379fb9818346c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:43:31.383701 23585 tablet_replica.cc:330] T 261235aa8c2b4dab95d7df22543c0576 P 639ff99251f5431c95a647cafe002de1: stopping tablet replica
I20260619 02:43:31.383831 23585 raft_consensus.cc:2229] T 261235aa8c2b4dab95d7df22543c0576 P 639ff99251f5431c95a647cafe002de1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:43:31.383941 23585 raft_consensus.cc:2258] T 261235aa8c2b4dab95d7df22543c0576 P 639ff99251f5431c95a647cafe002de1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:43:31.383936 23584 ts_tablet_manager.cc:1905] T 261235aa8c2b4dab95d7df22543c0576 P e05ddbd262bb4e6d9a4379fb9818346c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:43:31.384341 23585 ts_tablet_manager.cc:1905] T 261235aa8c2b4dab95d7df22543c0576 P 639ff99251f5431c95a647cafe002de1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:43:31.385537 23584 ts_tablet_manager.cc:1918] T 261235aa8c2b4dab95d7df22543c0576 P e05ddbd262bb4e6d9a4379fb9818346c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:43:31.385641 23584 log.cc:1196] T 261235aa8c2b4dab95d7df22543c0576 P e05ddbd262bb4e6d9a4379fb9818346c: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/ts-1/wal/wals/261235aa8c2b4dab95d7df22543c0576
I20260619 02:43:31.385898 23584 ts_tablet_manager.cc:1939] T 261235aa8c2b4dab95d7df22543c0576 P e05ddbd262bb4e6d9a4379fb9818346c: Deleting consensus metadata
I20260619 02:43:31.386282 23585 ts_tablet_manager.cc:1918] T 261235aa8c2b4dab95d7df22543c0576 P 639ff99251f5431c95a647cafe002de1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:43:31.386358 23585 log.cc:1196] T 261235aa8c2b4dab95d7df22543c0576 P 639ff99251f5431c95a647cafe002de1: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836772852275-6085-0/minicluster-data/ts-2/wal/wals/261235aa8c2b4dab95d7df22543c0576
I20260619 02:43:31.386576 23585 ts_tablet_manager.cc:1939] T 261235aa8c2b4dab95d7df22543c0576 P 639ff99251f5431c95a647cafe002de1: Deleting consensus metadata
I20260619 02:43:31.386924 23071 catalog_manager.cc:4841] TS e05ddbd262bb4e6d9a4379fb9818346c (127.5.241.66:46325): tablet 261235aa8c2b4dab95d7df22543c0576 (table test-table [id=3299ab9ac3754a639e6a6d56d26dfa3a]) successfully deleted
I20260619 02:43:31.387028 23071 catalog_manager.cc:4841] TS 639ff99251f5431c95a647cafe002de1 (127.5.241.67:34709): tablet 261235aa8c2b4dab95d7df22543c0576 (table test-table [id=3299ab9ac3754a639e6a6d56d26dfa3a]) successfully deleted
W20260619 02:43:31.388619 23074 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.241.65:37883 (error 108)
I20260619 02:43:31.388831 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 23283
I20260619 02:43:31.393697 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 23417
I20260619 02:43:31.393733 23554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:31.393587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:37883 (local address 127.0.0.1:33614)
0619 02:43:31.393654 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:31.393697 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.65:37883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260619 02:43:31.393932 23074 catalog_manager.cc:4584] TS df17d8367f794c0290ba6bce5c21f435 (127.5.241.65:37883): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 261235aa8c2b4dab95d7df22543c0576: Network error: Client connection negotiation failed: client connection to 127.5.241.65:37883: connect: Connection refused (error 111)
I20260619 02:43:31.397892 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 23055
2026-06-19T02:43:31Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2273 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-06-19T02:43:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:43:31Z Disabled control of system clock
I20260619 02:43:31.424499 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:43683
--webserver_interface=127.5.241.126
--webserver_port=0
--builtin_ntp_servers=127.5.241.84:33015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:43683
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 02:43:31.503218 23590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:31.503446 23590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:31.503468 23590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:31.504654 23590 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:43:31.504698 23590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:31.504710 23590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:31.504724 23590 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:43:31.504741 23590 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:43:31.506100 23590 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:33015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:43683
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:43683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.241.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23590
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:43:31.506356 23590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:31.506645 23590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:31.511466 23598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:31.511466 23596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:31.511480 23595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:31.511762 23590 server_base.cc:1034] running on GCE node
I20260619 02:43:31.512080 23590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:31.512328 23590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:31.513463 23590 hybrid_clock.cc:648] HybridClock initialized: now 1781837011513449 us; error 30 us; skew 500 ppm
I20260619 02:43:31.514803 23590 webserver.cc:460] Webserver started at http://127.5.241.126:36521/ using document root <none> and password file <none>
I20260619 02:43:31.515085 23590 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:31.515136 23590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:31.515321 23590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:43:31.516450 23590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/master-0/data/instance:
uuid: "9fa66084103a46f0a12262b14a1fd2e2"
format_stamp: "Formatted at 2026-06-19 02:43:31 on dist-test-slave-zpbg"
server_key: "62dce5237d554ef8d18d5883b21db9e9"
server_key_iv: "d8c1edd5f6d487fada515777f729067f"
server_key_version: "encryptionkey@0"
I20260619 02:43:31.516851 23590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/master-0/wal/instance:
uuid: "9fa66084103a46f0a12262b14a1fd2e2"
format_stamp: "Formatted at 2026-06-19 02:43:31 on dist-test-slave-zpbg"
server_key: "62dce5237d554ef8d18d5883b21db9e9"
server_key_iv: "d8c1edd5f6d487fada515777f729067f"
server_key_version: "encryptionkey@0"
I20260619 02:43:31.518314 23590 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:43:31.519060 23604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:31.519232 23590 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:43:31.519323 23590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "9fa66084103a46f0a12262b14a1fd2e2"
format_stamp: "Formatted at 2026-06-19 02:43:31 on dist-test-slave-zpbg"
server_key: "62dce5237d554ef8d18d5883b21db9e9"
server_key_iv: "d8c1edd5f6d487fada515777f729067f"
server_key_version: "encryptionkey@0"
I20260619 02:43:31.519397 23590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:31.567585 23590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:31.567932 23590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:31.568091 23590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:31.571941 23590 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.126:43683
I20260619 02:43:31.571997 23656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.126:43683 every 8 connection(s)
I20260619 02:43:31.572351 23590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:43:31.573323 23657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:31.575564 23657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fa66084103a46f0a12262b14a1fd2e2: Bootstrap starting.
I20260619 02:43:31.576117 23657 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9fa66084103a46f0a12262b14a1fd2e2: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:31.576411 23657 log.cc:824] T 00000000000000000000000000000000 P 9fa66084103a46f0a12262b14a1fd2e2: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:31.577055 23657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fa66084103a46f0a12262b14a1fd2e2: No bootstrap required, opened a new log
I20260619 02:43:31.578622 23657 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9fa66084103a46f0a12262b14a1fd2e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fa66084103a46f0a12262b14a1fd2e2" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 43683 } }
I20260619 02:43:31.578753 23657 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9fa66084103a46f0a12262b14a1fd2e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:31.578840 23657 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9fa66084103a46f0a12262b14a1fd2e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fa66084103a46f0a12262b14a1fd2e2, State: Initialized, Role: FOLLOWER
I20260619 02:43:31.578980 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 23590
I20260619 02:43:31.579002 23657 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9fa66084103a46f0a12262b14a1fd2e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fa66084103a46f0a12262b14a1fd2e2" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 43683 } }
I20260619 02:43:31.579097 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/master-0/wal/instance
I20260619 02:43:31.579098 23657 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9fa66084103a46f0a12262b14a1fd2e2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:43:31.579135 23657 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9fa66084103a46f0a12262b14a1fd2e2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:43:31.579190 23657 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9fa66084103a46f0a12262b14a1fd2e2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:31.579315 6085 external_mini_cluster.cc:1428] Setting key 48f6cf09577f64d2fba772a9983793c3
I20260619 02:43:31.579865 23657 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9fa66084103a46f0a12262b14a1fd2e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fa66084103a46f0a12262b14a1fd2e2" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 43683 } }
I20260619 02:43:31.580047 23657 leader_election.cc:302] T 00000000000000000000000000000000 P 9fa66084103a46f0a12262b14a1fd2e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9fa66084103a46f0a12262b14a1fd2e2; no voters:
I20260619 02:43:31.580209 23657 leader_election.cc:288] T 00000000000000000000000000000000 P 9fa66084103a46f0a12262b14a1fd2e2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:43:31.580242 23662 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9fa66084103a46f0a12262b14a1fd2e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:43:31.580363 23662 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9fa66084103a46f0a12262b14a1fd2e2 [term 1 LEADER]: Becoming Leader. State: Replica: 9fa66084103a46f0a12262b14a1fd2e2, State: Running, Role: LEADER
I20260619 02:43:31.580468 23657 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9fa66084103a46f0a12262b14a1fd2e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:43:31.580511 23662 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9fa66084103a46f0a12262b14a1fd2e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fa66084103a46f0a12262b14a1fd2e2" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 43683 } }
I20260619 02:43:31.581007 23663 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fa66084103a46f0a12262b14a1fd2e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9fa66084103a46f0a12262b14a1fd2e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fa66084103a46f0a12262b14a1fd2e2" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 43683 } } }
I20260619 02:43:31.581110 23663 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fa66084103a46f0a12262b14a1fd2e2 [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:31.581300 23664 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fa66084103a46f0a12262b14a1fd2e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9fa66084103a46f0a12262b14a1fd2e2. Latest consensus state: current_term: 1 leader_uuid: "9fa66084103a46f0a12262b14a1fd2e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fa66084103a46f0a12262b14a1fd2e2" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 43683 } } }
I20260619 02:43:31.581409 23664 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fa66084103a46f0a12262b14a1fd2e2 [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:31.581310 23671 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:43:31.581959 23671 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:43:31.583573 23671 catalog_manager.cc:1347] Generated new cluster ID: df84cd1869be4e858c7fcb27d59883fa
I20260619 02:43:31.583624 23671 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:43:31.586052 23661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:31.579821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37114 (local address 127.5.241.126:43683)
0619 02:43:31.580040 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:31.580048 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:31.580071 (+ 23us) server_negotiation.cc:407] Connection header received
0619 02:43:31.580196 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:31.580214 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:31.580290 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:31.580689 (+ 399us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:31.580781 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:31.582645 (+ 1864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:31.583778 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:31.584385 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:31.584513 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:31.585247 (+ 734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:31.585252 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:31.585252 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:43:31.585462 (+ 210us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:31.585505 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:43:31.585522 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:31.585593 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:31.585663 (+ 70us) server_negotiation.cc:299] Negotiation successful
0619 02:43:31.585845 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":107,"threads_started":1}
I20260619 02:43:31.595443 23671 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:43:31.595921 23671 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:43:31.602463 23671 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9fa66084103a46f0a12262b14a1fd2e2: Generated new TSK 0
I20260619 02:43:31.602650 23671 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:43:31.604920 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.65:0
--local_ip_for_outbound_sockets=127.5.241.65
--webserver_interface=127.5.241.65
--webserver_port=0
--tserver_master_addrs=127.5.241.126:43683
--builtin_ntp_servers=127.5.241.84:33015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 02:43:31.701761 23681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:31.701965 23681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:31.701988 23681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:31.703176 23681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:31.703214 23681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:31.703240 23681 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.65
I20260619 02:43:31.704617 23681 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:33015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.241.65
--webserver_port=0
--tserver_master_addrs=127.5.241.126:43683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.65
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:43:31.704840 23681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:31.705153 23681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:31.711288 23687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:31.711572 23686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:31.711583 23689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:31.711796 23681 server_base.cc:1034] running on GCE node
I20260619 02:43:31.712029 23681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:31.712323 23681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:31.713506 23681 hybrid_clock.cc:648] HybridClock initialized: now 1781837011713485 us; error 36 us; skew 500 ppm
I20260619 02:43:31.715037 23681 webserver.cc:460] Webserver started at http://127.5.241.65:46463/ using document root <none> and password file <none>
I20260619 02:43:31.715339 23681 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:31.715409 23681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:31.715539 23681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:43:31.716974 23681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/ts-0/data/instance:
uuid: "0fcd2bec719f400d88d330ed079857e6"
format_stamp: "Formatted at 2026-06-19 02:43:31 on dist-test-slave-zpbg"
server_key: "e9cf801c32383af365a9707bb54f3a31"
server_key_iv: "0ccc4db26d122235d16352d2542142b9"
server_key_version: "encryptionkey@0"
I20260619 02:43:31.717392 23681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/ts-0/wal/instance:
uuid: "0fcd2bec719f400d88d330ed079857e6"
format_stamp: "Formatted at 2026-06-19 02:43:31 on dist-test-slave-zpbg"
server_key: "e9cf801c32383af365a9707bb54f3a31"
server_key_iv: "0ccc4db26d122235d16352d2542142b9"
server_key_version: "encryptionkey@0"
I20260619 02:43:31.718981 23681 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 02:43:31.719844 23695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:31.720054 23681 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:43:31.720144 23681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/ts-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/ts-0/wal
uuid: "0fcd2bec719f400d88d330ed079857e6"
format_stamp: "Formatted at 2026-06-19 02:43:31 on dist-test-slave-zpbg"
server_key: "e9cf801c32383af365a9707bb54f3a31"
server_key_iv: "0ccc4db26d122235d16352d2542142b9"
server_key_version: "encryptionkey@0"
I20260619 02:43:31.720218 23681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:31.740250 23681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:31.740594 23681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:31.740756 23681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:31.741019 23681 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:43:31.741360 23681 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:43:31.741393 23681 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:31.741416 23681 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:43:31.741429 23681 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:31.747205 23681 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.65:35855
I20260619 02:43:31.747244 23808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.65:35855 every 8 connection(s)
I20260619 02:43:31.747581 23681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
I20260619 02:43:31.750609 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 23681
I20260619 02:43:31.750741 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/ts-0/wal/instance
I20260619 02:43:31.750980 6085 external_mini_cluster.cc:1428] Setting key c3e5aa36181210d94f835a519f65101b
I20260619 02:43:31.751927 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.66:0
--local_ip_for_outbound_sockets=127.5.241.66
--webserver_interface=127.5.241.66
--webserver_port=0
--tserver_master_addrs=127.5.241.126:43683
--builtin_ntp_servers=127.5.241.84:33015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 02:43:31.753199 23661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:31.748368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:54667 (local address 127.5.241.126:43683)
0619 02:43:31.748412 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:31.748414 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:31.748662 (+ 248us) server_negotiation.cc:407] Connection header received
0619 02:43:31.749121 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:31.749122 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:31.749136 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:31.749205 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:31.749856 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:31.750459 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:31.751691 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:31.751913 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:31.752002 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:31.752895 (+ 893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:31.752897 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:31.752897 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:43:31.752904 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:31.752920 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:43:31.752924 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:31.752993 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:31.753089 (+ 96us) server_negotiation.cc:299] Negotiation successful
0619 02:43:31.753121 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 02:43:31.753432 23811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:31.748352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:43683 (local address 127.5.241.65:54667)
0619 02:43:31.748535 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:31.748567 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:31.749028 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:31.749262 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:31.749265 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:31.749273 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:43:31.749758 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:31.749762 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:31.750553 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:31.750554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:31.751595 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:31.751598 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:31.752012 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:31.752014 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:31.752154 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:31.752754 (+ 600us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:43:31.752755 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:43:31.752782 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:43:31.752805 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:43:31.752991 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:43:31.752997 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:43:31.753000 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:43:31.753092 (+ 92us) client_negotiation.cc:240] Negotiation successful
0619 02:43:31.753192 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":102,"threads_started":1}
I20260619 02:43:31.753958 23809 heartbeater.cc:347] Connected to a master server at 127.5.241.126:43683
I20260619 02:43:31.754037 23809 heartbeater.cc:464] Registering TS with master...
I20260619 02:43:31.754236 23809 heartbeater.cc:511] Master 127.5.241.126:43683 requested a full tablet report, sending...
I20260619 02:43:31.754875 23621 ts_manager.cc:194] Registered new tserver with Master: 0fcd2bec719f400d88d330ed079857e6 (127.5.241.65:35855)
I20260619 02:43:31.755484 23621 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.241.65:54667
W20260619 02:43:31.832433 23812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:31.832643 23812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:31.832664 23812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:31.833827 23812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:31.833865 23812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:31.833889 23812 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.66
I20260619 02:43:31.835220 23812 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:33015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.241.66
--webserver_port=0
--tserver_master_addrs=127.5.241.126:43683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23812
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.66
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:43:31.835450 23812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:31.835736 23812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:31.840720 23818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:31.840731 23817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:31.840806 23820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:31.840871 23812 server_base.cc:1034] running on GCE node
I20260619 02:43:31.841182 23812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:31.841511 23812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:31.842696 23812 hybrid_clock.cc:648] HybridClock initialized: now 1781837011842685 us; error 29 us; skew 500 ppm
I20260619 02:43:31.844125 23812 webserver.cc:460] Webserver started at http://127.5.241.66:36677/ using document root <none> and password file <none>
I20260619 02:43:31.844367 23812 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:31.844414 23812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:31.844583 23812 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:43:31.845641 23812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/ts-1/data/instance:
uuid: "02f9abe25a234b66a6509e002272b4e4"
format_stamp: "Formatted at 2026-06-19 02:43:31 on dist-test-slave-zpbg"
server_key: "99698cf502d5bc7cb78ef0861804fb07"
server_key_iv: "26a2989eb1d62e9a3237bd1f6d25ed15"
server_key_version: "encryptionkey@0"
I20260619 02:43:31.845999 23812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/ts-1/wal/instance:
uuid: "02f9abe25a234b66a6509e002272b4e4"
format_stamp: "Formatted at 2026-06-19 02:43:31 on dist-test-slave-zpbg"
server_key: "99698cf502d5bc7cb78ef0861804fb07"
server_key_iv: "26a2989eb1d62e9a3237bd1f6d25ed15"
server_key_version: "encryptionkey@0"
I20260619 02:43:31.847390 23812 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:43:31.848100 23826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:31.848274 23812 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:43:31.848359 23812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/ts-1/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/ts-1/wal
uuid: "02f9abe25a234b66a6509e002272b4e4"
format_stamp: "Formatted at 2026-06-19 02:43:31 on dist-test-slave-zpbg"
server_key: "99698cf502d5bc7cb78ef0861804fb07"
server_key_iv: "26a2989eb1d62e9a3237bd1f6d25ed15"
server_key_version: "encryptionkey@0"
I20260619 02:43:31.848429 23812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:31.861558 23812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:31.861864 23812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:31.862010 23812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:31.862242 23812 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:43:31.862572 23812 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:43:31.862627 23812 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:31.862668 23812 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:43:31.862695 23812 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:31.868363 23812 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.66:35891
I20260619 02:43:31.868404 23939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.66:35891 every 8 connection(s)
I20260619 02:43:31.868760 23812 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
I20260619 02:43:31.873781 23661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:31.869576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:36997 (local address 127.5.241.126:43683)
0619 02:43:31.869650 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:31.869652 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:31.869822 (+ 170us) server_negotiation.cc:407] Connection header received
0619 02:43:31.870313 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:31.870314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:31.870323 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:31.870383 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:31.871098 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:31.871575 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:31.872492 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:31.872663 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:31.872712 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:31.873496 (+ 784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:31.873497 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:31.873497 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:43:31.873502 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:31.873517 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:43:31.873527 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:31.873587 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:31.873682 (+ 95us) server_negotiation.cc:299] Negotiation successful
0619 02:43:31.873706 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 02:43:31.873917 23942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:31.869495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:43683 (local address 127.5.241.66:36997)
0619 02:43:31.869696 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:31.869737 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:31.870230 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:31.870449 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:31.870453 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:31.870460 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:43:31.871024 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:31.871030 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:31.871654 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:31.871654 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:31.872420 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:31.872422 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:31.872737 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:31.872737 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:31.872842 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:31.873347 (+ 505us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:43:31.873347 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:43:31.873373 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:43:31.873394 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:43:31.873594 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:43:31.873598 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:43:31.873600 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 02:43:31.873653 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 02:43:31.873733 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":92,"threads_started":1}
I20260619 02:43:31.874370 23940 heartbeater.cc:347] Connected to a master server at 127.5.241.126:43683
I20260619 02:43:31.874459 23940 heartbeater.cc:464] Registering TS with master...
I20260619 02:43:31.874590 23940 heartbeater.cc:511] Master 127.5.241.126:43683 requested a full tablet report, sending...
I20260619 02:43:31.874922 23621 ts_manager.cc:194] Registered new tserver with Master: 02f9abe25a234b66a6509e002272b4e4 (127.5.241.66:35891)
I20260619 02:43:31.875280 23621 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.241.66:36997
I20260619 02:43:31.877692 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 23812
I20260619 02:43:31.877830 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/ts-1/wal/instance
I20260619 02:43:31.878048 6085 external_mini_cluster.cc:1428] Setting key b343a6df28ff96569da4daac322ed12d
I20260619 02:43:31.878983 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.67:0
--local_ip_for_outbound_sockets=127.5.241.67
--webserver_interface=127.5.241.67
--webserver_port=0
--tserver_master_addrs=127.5.241.126:43683
--builtin_ntp_servers=127.5.241.84:33015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 02:43:31.958559 23943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:31.958756 23943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:31.958839 23943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:31.960001 23943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:31.960043 23943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:31.960068 23943 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.67
I20260619 02:43:31.961397 23943 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:33015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.241.67
--webserver_port=0
--tserver_master_addrs=127.5.241.126:43683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23943
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.67
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:43:31.961644 23943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:31.961931 23943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:31.967068 23948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:31.967068 23949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:31.967111 23951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:31.967280 23943 server_base.cc:1034] running on GCE node
I20260619 02:43:31.967502 23943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:31.967689 23943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:31.968816 23943 hybrid_clock.cc:648] HybridClock initialized: now 1781837011968804 us; error 30 us; skew 500 ppm
I20260619 02:43:31.969929 23943 webserver.cc:460] Webserver started at http://127.5.241.67:40729/ using document root <none> and password file <none>
I20260619 02:43:31.970120 23943 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:31.970160 23943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:31.970260 23943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:43:31.971360 23943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/ts-2/data/instance:
uuid: "9db1c960a0064ef9a8d5f94a56f313c0"
format_stamp: "Formatted at 2026-06-19 02:43:31 on dist-test-slave-zpbg"
server_key: "a95cfbc323012bfd36a31c341f51af12"
server_key_iv: "eb3f46129c40a1dc81bedce00574543a"
server_key_version: "encryptionkey@0"
I20260619 02:43:31.971653 23943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/ts-2/wal/instance:
uuid: "9db1c960a0064ef9a8d5f94a56f313c0"
format_stamp: "Formatted at 2026-06-19 02:43:31 on dist-test-slave-zpbg"
server_key: "a95cfbc323012bfd36a31c341f51af12"
server_key_iv: "eb3f46129c40a1dc81bedce00574543a"
server_key_version: "encryptionkey@0"
I20260619 02:43:31.972822 23943 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:43:31.973503 23957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:31.973655 23943 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:43:31.973723 23943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/ts-2/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/ts-2/wal
uuid: "9db1c960a0064ef9a8d5f94a56f313c0"
format_stamp: "Formatted at 2026-06-19 02:43:31 on dist-test-slave-zpbg"
server_key: "a95cfbc323012bfd36a31c341f51af12"
server_key_iv: "eb3f46129c40a1dc81bedce00574543a"
server_key_version: "encryptionkey@0"
I20260619 02:43:31.973776 23943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:31.988929 23943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:31.989220 23943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:31.989336 23943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:31.989557 23943 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:43:31.989946 23943 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:43:31.989990 23943 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:31.990015 23943 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:43:31.990073 23943 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:31.996392 23943 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.67:42513
I20260619 02:43:31.996433 24070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.67:42513 every 8 connection(s)
I20260619 02:43:31.996796 23943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
I20260619 02:43:32.002008 23661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:31.997605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:53545 (local address 127.5.241.126:43683)
0619 02:43:31.997669 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:31.997673 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:31.997937 (+ 264us) server_negotiation.cc:407] Connection header received
0619 02:43:31.998461 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:31.998462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:31.998471 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:31.998536 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:31.999239 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:31.999729 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.000768 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.000957 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.001008 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:32.001790 (+ 782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:32.001791 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:32.001791 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:43:32.001795 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:32.001809 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:43:32.001812 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:32.001854 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:32.001923 (+ 69us) server_negotiation.cc:299] Negotiation successful
0619 02:43:32.001949 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 02:43:32.002255 24073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:31.997598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:43683 (local address 127.5.241.67:53545)
0619 02:43:31.997794 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:31.997838 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:31.998372 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:31.998599 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:31.998602 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:31.998611 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:43:31.999163 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:31.999168 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:31.999818 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:31.999819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:32.000687 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:32.000689 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.001036 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.001037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:32.001155 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:32.001653 (+ 498us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:43:32.001653 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:43:32.001682 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:43:32.001705 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:43:32.001856 (+ 151us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:43:32.001858 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:43:32.001860 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 02:43:32.001910 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 02:43:32.001987 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":97,"threads_started":1}
I20260619 02:43:32.002887 24071 heartbeater.cc:347] Connected to a master server at 127.5.241.126:43683
I20260619 02:43:32.003001 24071 heartbeater.cc:464] Registering TS with master...
I20260619 02:43:32.003185 24071 heartbeater.cc:511] Master 127.5.241.126:43683 requested a full tablet report, sending...
I20260619 02:43:32.003561 23621 ts_manager.cc:194] Registered new tserver with Master: 9db1c960a0064ef9a8d5f94a56f313c0 (127.5.241.67:42513)
I20260619 02:43:32.003845 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 23943
I20260619 02:43:32.003896 23621 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.241.67:53545
I20260619 02:43:32.003969 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/ts-2/wal/instance
I20260619 02:43:32.004180 6085 external_mini_cluster.cc:1428] Setting key 8376d1e9092b01d71c89361e357b8538
I20260619 02:43:32.005028 6085 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:43:32.008946 23661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:32.006343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37122 (local address 127.5.241.126:43683)
0619 02:43:32.006392 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:32.006393 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:32.006494 (+ 101us) server_negotiation.cc:407] Connection header received
0619 02:43:32.006533 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:32.006533 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:32.006542 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:32.006590 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:32.006758 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.007212 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.008011 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.008187 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.008253 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:32.008777 (+ 524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:32.008778 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:32.008779 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:43:32.008782 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:32.008793 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:43:32.008797 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:32.008840 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:32.008875 (+ 35us) server_negotiation.cc:299] Negotiation successful
0619 02:43:32.008894 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 02:43:32.010355 23621 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37122:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:43:32.011953 23621 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:43:32.019301 24082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:32.014369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.66:35891 (local address 127.0.0.1:38998)
0619 02:43:32.014581 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:32.014593 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:32.014670 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:32.015034 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:32.015037 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:32.015040 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:43:32.015110 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:32.015112 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.016547 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.016548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:32.017985 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:32.017988 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.018964 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.018966 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:32.019045 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:32.019061 (+ 16us) client_negotiation.cc:769] Sending connection context
0619 02:43:32.019141 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 02:43:32.019210 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":62,"thread_start_us":93,"threads_started":1}
I20260619 02:43:32.019299 24083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:32.014456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38998 (local address 127.5.241.66:35891)
0619 02:43:32.014828 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:32.014831 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:32.014840 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:43:32.014879 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:32.014881 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:32.014896 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:32.014977 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:43:32.015319 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.016415 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.018082 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.018881 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.018954 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:32.019058 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:32.019167 (+ 109us) server_negotiation.cc:299] Negotiation successful
0619 02:43:32.019208 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":89,"threads_started":1}
I20260619 02:43:32.019466 24079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:32.014429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.67:42513 (local address 127.0.0.1:56756)
0619 02:43:32.014590 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:32.014600 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:32.014662 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:32.015159 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:32.015161 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:32.015163 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:43:32.015221 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:32.015223 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.017538 (+ 2315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.017539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:32.018468 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:32.018471 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.019280 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.019281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:32.019341 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:32.019345 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:43:32.019377 (+ 32us) client_negotiation.cc:240] Negotiation successful
0619 02:43:32.019414 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":10,"thread_start_us":105,"threads_started":1}
I20260619 02:43:32.019833 24084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:32.014697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56756 (local address 127.5.241.67:42513)
0619 02:43:32.014859 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:32.014863 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:32.014872 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:43:32.014906 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:32.014910 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:32.014951 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:32.015147 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:43:32.016537 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.017461 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.018552 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.019188 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.019550 (+ 362us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:32.019645 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:32.019670 (+ 25us) server_negotiation.cc:299] Negotiation successful
0619 02:43:32.019730 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":100,"threads_started":1}
I20260619 02:43:32.020174 23874 tablet_service.cc:1467] Processing CreateTablet for tablet c0579e78105340acb1d53f0d717fcd4d (DEFAULT_TABLE table=test-table [id=8734f30e4eac438a8172397adcab3ef0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:43:32.020385 24080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:32.014241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:35855 (local address 127.0.0.1:58116)
0619 02:43:32.014546 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:32.014591 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:32.014659 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:32.015185 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:32.015186 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:32.015188 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:43:32.015227 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:32.015229 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.016547 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.016548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:32.017985 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:32.017988 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.020131 (+ 2143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.020132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:32.020200 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:32.020204 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:43:32.020272 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 02:43:32.020312 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"mutex_wait_us":42,"thread_start_us":57,"threads_started":1}
I20260619 02:43:32.020430 24081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:32.014356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58116 (local address 127.5.241.65:35855)
0619 02:43:32.014823 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:32.014829 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:32.014840 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:43:32.014879 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:32.014881 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:32.014896 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:32.014977 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:43:32.015288 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.016440 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.018849 (+ 2409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.019952 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.020018 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:32.020094 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:32.020300 (+ 206us) server_negotiation.cc:299] Negotiation successful
0619 02:43:32.020345 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":109,"threads_started":1}
I20260619 02:43:32.020443 24005 tablet_service.cc:1467] Processing CreateTablet for tablet c0579e78105340acb1d53f0d717fcd4d (DEFAULT_TABLE table=test-table [id=8734f30e4eac438a8172397adcab3ef0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:43:32.020586 23874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0579e78105340acb1d53f0d717fcd4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:32.020730 24005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0579e78105340acb1d53f0d717fcd4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:32.023427 24085 tablet_bootstrap.cc:492] T c0579e78105340acb1d53f0d717fcd4d P 9db1c960a0064ef9a8d5f94a56f313c0: Bootstrap starting.
I20260619 02:43:32.023758 24086 tablet_bootstrap.cc:492] T c0579e78105340acb1d53f0d717fcd4d P 02f9abe25a234b66a6509e002272b4e4: Bootstrap starting.
I20260619 02:43:32.024151 24085 tablet_bootstrap.cc:654] T c0579e78105340acb1d53f0d717fcd4d P 9db1c960a0064ef9a8d5f94a56f313c0: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:32.024228 23743 tablet_service.cc:1467] Processing CreateTablet for tablet c0579e78105340acb1d53f0d717fcd4d (DEFAULT_TABLE table=test-table [id=8734f30e4eac438a8172397adcab3ef0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:43:32.024384 24086 tablet_bootstrap.cc:654] T c0579e78105340acb1d53f0d717fcd4d P 02f9abe25a234b66a6509e002272b4e4: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:32.024472 24085 log.cc:824] T c0579e78105340acb1d53f0d717fcd4d P 9db1c960a0064ef9a8d5f94a56f313c0: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:32.024603 23743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0579e78105340acb1d53f0d717fcd4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:32.024674 24086 log.cc:824] T c0579e78105340acb1d53f0d717fcd4d P 02f9abe25a234b66a6509e002272b4e4: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:32.025414 24086 tablet_bootstrap.cc:492] T c0579e78105340acb1d53f0d717fcd4d P 02f9abe25a234b66a6509e002272b4e4: No bootstrap required, opened a new log
I20260619 02:43:32.025490 24086 ts_tablet_manager.cc:1397] T c0579e78105340acb1d53f0d717fcd4d P 02f9abe25a234b66a6509e002272b4e4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:43:32.028049 24089 tablet_bootstrap.cc:492] T c0579e78105340acb1d53f0d717fcd4d P 0fcd2bec719f400d88d330ed079857e6: Bootstrap starting.
I20260619 02:43:32.028127 24086 raft_consensus.cc:348] T c0579e78105340acb1d53f0d717fcd4d P 02f9abe25a234b66a6509e002272b4e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fcd2bec719f400d88d330ed079857e6" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 35855 } } peers { permanent_uuid: "9db1c960a0064ef9a8d5f94a56f313c0" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 42513 } } peers { permanent_uuid: "02f9abe25a234b66a6509e002272b4e4" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 35891 } }
I20260619 02:43:32.028307 24086 raft_consensus.cc:374] T c0579e78105340acb1d53f0d717fcd4d P 02f9abe25a234b66a6509e002272b4e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:32.028353 24086 raft_consensus.cc:729] T c0579e78105340acb1d53f0d717fcd4d P 02f9abe25a234b66a6509e002272b4e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02f9abe25a234b66a6509e002272b4e4, State: Initialized, Role: FOLLOWER
I20260619 02:43:32.028469 24086 consensus_queue.cc:260] T c0579e78105340acb1d53f0d717fcd4d P 02f9abe25a234b66a6509e002272b4e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fcd2bec719f400d88d330ed079857e6" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 35855 } } peers { permanent_uuid: "9db1c960a0064ef9a8d5f94a56f313c0" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 42513 } } peers { permanent_uuid: "02f9abe25a234b66a6509e002272b4e4" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 35891 } }
I20260619 02:43:32.028651 24089 tablet_bootstrap.cc:654] T c0579e78105340acb1d53f0d717fcd4d P 0fcd2bec719f400d88d330ed079857e6: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:32.028740 24086 ts_tablet_manager.cc:1428] T c0579e78105340acb1d53f0d717fcd4d P 02f9abe25a234b66a6509e002272b4e4: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260619 02:43:32.028805 23940 heartbeater.cc:503] Master 127.5.241.126:43683 was elected leader, sending a full tablet report...
I20260619 02:43:32.028957 24089 log.cc:824] T c0579e78105340acb1d53f0d717fcd4d P 0fcd2bec719f400d88d330ed079857e6: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:32.029287 24085 tablet_bootstrap.cc:492] T c0579e78105340acb1d53f0d717fcd4d P 9db1c960a0064ef9a8d5f94a56f313c0: No bootstrap required, opened a new log
I20260619 02:43:32.029345 24085 ts_tablet_manager.cc:1397] T c0579e78105340acb1d53f0d717fcd4d P 9db1c960a0064ef9a8d5f94a56f313c0: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260619 02:43:32.030012 24089 tablet_bootstrap.cc:492] T c0579e78105340acb1d53f0d717fcd4d P 0fcd2bec719f400d88d330ed079857e6: No bootstrap required, opened a new log
I20260619 02:43:32.030095 24089 ts_tablet_manager.cc:1397] T c0579e78105340acb1d53f0d717fcd4d P 0fcd2bec719f400d88d330ed079857e6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:43:32.032063 24085 raft_consensus.cc:348] T c0579e78105340acb1d53f0d717fcd4d P 9db1c960a0064ef9a8d5f94a56f313c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fcd2bec719f400d88d330ed079857e6" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 35855 } } peers { permanent_uuid: "9db1c960a0064ef9a8d5f94a56f313c0" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 42513 } } peers { permanent_uuid: "02f9abe25a234b66a6509e002272b4e4" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 35891 } }
I20260619 02:43:32.032243 24085 raft_consensus.cc:374] T c0579e78105340acb1d53f0d717fcd4d P 9db1c960a0064ef9a8d5f94a56f313c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:32.032279 24085 raft_consensus.cc:729] T c0579e78105340acb1d53f0d717fcd4d P 9db1c960a0064ef9a8d5f94a56f313c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9db1c960a0064ef9a8d5f94a56f313c0, State: Initialized, Role: FOLLOWER
I20260619 02:43:32.032388 24085 consensus_queue.cc:260] T c0579e78105340acb1d53f0d717fcd4d P 9db1c960a0064ef9a8d5f94a56f313c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fcd2bec719f400d88d330ed079857e6" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 35855 } } peers { permanent_uuid: "9db1c960a0064ef9a8d5f94a56f313c0" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 42513 } } peers { permanent_uuid: "02f9abe25a234b66a6509e002272b4e4" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 35891 } }
I20260619 02:43:32.032634 24085 ts_tablet_manager.cc:1428] T c0579e78105340acb1d53f0d717fcd4d P 9db1c960a0064ef9a8d5f94a56f313c0: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260619 02:43:32.032681 24071 heartbeater.cc:503] Master 127.5.241.126:43683 was elected leader, sending a full tablet report...
I20260619 02:43:32.032610 24089 raft_consensus.cc:348] T c0579e78105340acb1d53f0d717fcd4d P 0fcd2bec719f400d88d330ed079857e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fcd2bec719f400d88d330ed079857e6" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 35855 } } peers { permanent_uuid: "9db1c960a0064ef9a8d5f94a56f313c0" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 42513 } } peers { permanent_uuid: "02f9abe25a234b66a6509e002272b4e4" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 35891 } }
I20260619 02:43:32.032756 24089 raft_consensus.cc:374] T c0579e78105340acb1d53f0d717fcd4d P 0fcd2bec719f400d88d330ed079857e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:32.032799 24089 raft_consensus.cc:729] T c0579e78105340acb1d53f0d717fcd4d P 0fcd2bec719f400d88d330ed079857e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fcd2bec719f400d88d330ed079857e6, State: Initialized, Role: FOLLOWER
I20260619 02:43:32.032904 24089 consensus_queue.cc:260] T c0579e78105340acb1d53f0d717fcd4d P 0fcd2bec719f400d88d330ed079857e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fcd2bec719f400d88d330ed079857e6" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 35855 } } peers { permanent_uuid: "9db1c960a0064ef9a8d5f94a56f313c0" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 42513 } } peers { permanent_uuid: "02f9abe25a234b66a6509e002272b4e4" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 35891 } }
I20260619 02:43:32.033177 24089 ts_tablet_manager.cc:1428] T c0579e78105340acb1d53f0d717fcd4d P 0fcd2bec719f400d88d330ed079857e6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 02:43:32.033205 23809 heartbeater.cc:503] Master 127.5.241.126:43683 was elected leader, sending a full tablet report...
W20260619 02:43:32.119304 23941 tablet.cc:2369] T c0579e78105340acb1d53f0d717fcd4d P 02f9abe25a234b66a6509e002272b4e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:43:32.218653 24093 raft_consensus.cc:482] T c0579e78105340acb1d53f0d717fcd4d P 0fcd2bec719f400d88d330ed079857e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:43:32.218883 24093 raft_consensus.cc:504] T c0579e78105340acb1d53f0d717fcd4d P 0fcd2bec719f400d88d330ed079857e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fcd2bec719f400d88d330ed079857e6" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 35855 } } peers { permanent_uuid: "9db1c960a0064ef9a8d5f94a56f313c0" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 42513 } } peers { permanent_uuid: "02f9abe25a234b66a6509e002272b4e4" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 35891 } }
I20260619 02:43:32.219298 24093 leader_election.cc:288] T c0579e78105340acb1d53f0d717fcd4d P 0fcd2bec719f400d88d330ed079857e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9db1c960a0064ef9a8d5f94a56f313c0 (127.5.241.67:42513), 02f9abe25a234b66a6509e002272b4e4 (127.5.241.66:35891)
I20260619 02:43:32.223001 24083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:32.219566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:38263 (local address 127.5.241.66:35891)
0619 02:43:32.219608 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:32.219610 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:32.219813 (+ 203us) server_negotiation.cc:407] Connection header received
0619 02:43:32.219841 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:32.219842 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:32.219849 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:32.219922 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:43:32.220037 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.220727 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.222053 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.222629 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.222699 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:32.222707 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:32.222879 (+ 172us) server_negotiation.cc:299] Negotiation successful
0619 02:43:32.222920 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 02:43:32.223007 24084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:32.219536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:56503 (local address 127.5.241.67:42513)
0619 02:43:32.219589 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:32.219591 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:32.219601 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:43:32.219634 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:32.219634 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:32.219643 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:32.219704 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:43:32.219893 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.220634 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.222053 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.222624 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.222699 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:32.222707 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:32.222891 (+ 184us) server_negotiation.cc:299] Negotiation successful
0619 02:43:32.222930 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 02:43:32.223002 23811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:32.219427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.67:42513 (local address 127.5.241.65:56503)
0619 02:43:32.219491 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:32.219505 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:32.219554 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:32.219733 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:32.219734 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:32.219736 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:43:32.219814 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:32.219816 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.220720 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.220721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:32.221965 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:32.221968 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.222710 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.222712 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:32.222805 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:32.222811 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 02:43:32.222865 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 02:43:32.222908 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":29}
I20260619 02:43:32.223008 24094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:32.219510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.66:35891 (local address 127.5.241.65:38263)
0619 02:43:32.219655 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:32.219664 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:32.219755 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:32.219940 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:32.219940 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:32.219942 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:43:32.219992 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:32.219993 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.220800 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.220801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:32.221965 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:32.221967 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.222722 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.222723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:32.222817 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:32.222821 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:43:32.222871 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 02:43:32.222908 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":2,"thread_start_us":105,"threads_started":1}
I20260619 02:43:32.223390 24025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0579e78105340acb1d53f0d717fcd4d" candidate_uuid: "0fcd2bec719f400d88d330ed079857e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9db1c960a0064ef9a8d5f94a56f313c0" is_pre_election: true
I20260619 02:43:32.223423 23894 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0579e78105340acb1d53f0d717fcd4d" candidate_uuid: "0fcd2bec719f400d88d330ed079857e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02f9abe25a234b66a6509e002272b4e4" is_pre_election: true
I20260619 02:43:32.223555 24025 raft_consensus.cc:2454] T c0579e78105340acb1d53f0d717fcd4d P 9db1c960a0064ef9a8d5f94a56f313c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fcd2bec719f400d88d330ed079857e6 in term 0.
I20260619 02:43:32.223575 23894 raft_consensus.cc:2454] T c0579e78105340acb1d53f0d717fcd4d P 02f9abe25a234b66a6509e002272b4e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fcd2bec719f400d88d330ed079857e6 in term 0.
I20260619 02:43:32.223812 23698 leader_election.cc:302] T c0579e78105340acb1d53f0d717fcd4d P 0fcd2bec719f400d88d330ed079857e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02f9abe25a234b66a6509e002272b4e4, 0fcd2bec719f400d88d330ed079857e6; no voters:
I20260619 02:43:32.223927 24093 raft_consensus.cc:2781] T c0579e78105340acb1d53f0d717fcd4d P 0fcd2bec719f400d88d330ed079857e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:43:32.223979 24093 raft_consensus.cc:482] T c0579e78105340acb1d53f0d717fcd4d P 0fcd2bec719f400d88d330ed079857e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:43:32.224012 24093 raft_consensus.cc:3037] T c0579e78105340acb1d53f0d717fcd4d P 0fcd2bec719f400d88d330ed079857e6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:32.224781 24093 raft_consensus.cc:504] T c0579e78105340acb1d53f0d717fcd4d P 0fcd2bec719f400d88d330ed079857e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fcd2bec719f400d88d330ed079857e6" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 35855 } } peers { permanent_uuid: "9db1c960a0064ef9a8d5f94a56f313c0" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 42513 } } peers { permanent_uuid: "02f9abe25a234b66a6509e002272b4e4" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 35891 } }
I20260619 02:43:32.224920 24093 leader_election.cc:288] T c0579e78105340acb1d53f0d717fcd4d P 0fcd2bec719f400d88d330ed079857e6 [CANDIDATE]: Term 1 election: Requested vote from peers 9db1c960a0064ef9a8d5f94a56f313c0 (127.5.241.67:42513), 02f9abe25a234b66a6509e002272b4e4 (127.5.241.66:35891)
I20260619 02:43:32.225060 23894 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0579e78105340acb1d53f0d717fcd4d" candidate_uuid: "0fcd2bec719f400d88d330ed079857e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02f9abe25a234b66a6509e002272b4e4"
I20260619 02:43:32.225060 24025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0579e78105340acb1d53f0d717fcd4d" candidate_uuid: "0fcd2bec719f400d88d330ed079857e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9db1c960a0064ef9a8d5f94a56f313c0"
I20260619 02:43:32.225136 24025 raft_consensus.cc:3037] T c0579e78105340acb1d53f0d717fcd4d P 9db1c960a0064ef9a8d5f94a56f313c0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:32.225136 23894 raft_consensus.cc:3037] T c0579e78105340acb1d53f0d717fcd4d P 02f9abe25a234b66a6509e002272b4e4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:32.226006 23894 raft_consensus.cc:2454] T c0579e78105340acb1d53f0d717fcd4d P 02f9abe25a234b66a6509e002272b4e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fcd2bec719f400d88d330ed079857e6 in term 1.
I20260619 02:43:32.226006 24025 raft_consensus.cc:2454] T c0579e78105340acb1d53f0d717fcd4d P 9db1c960a0064ef9a8d5f94a56f313c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fcd2bec719f400d88d330ed079857e6 in term 1.
I20260619 02:43:32.226202 23698 leader_election.cc:302] T c0579e78105340acb1d53f0d717fcd4d P 0fcd2bec719f400d88d330ed079857e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02f9abe25a234b66a6509e002272b4e4, 0fcd2bec719f400d88d330ed079857e6; no voters:
I20260619 02:43:32.226301 24093 raft_consensus.cc:2781] T c0579e78105340acb1d53f0d717fcd4d P 0fcd2bec719f400d88d330ed079857e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:43:32.226424 24093 raft_consensus.cc:686] T c0579e78105340acb1d53f0d717fcd4d P 0fcd2bec719f400d88d330ed079857e6 [term 1 LEADER]: Becoming Leader. State: Replica: 0fcd2bec719f400d88d330ed079857e6, State: Running, Role: LEADER
I20260619 02:43:32.226493 24093 consensus_queue.cc:237] T c0579e78105340acb1d53f0d717fcd4d P 0fcd2bec719f400d88d330ed079857e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fcd2bec719f400d88d330ed079857e6" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 35855 } } peers { permanent_uuid: "9db1c960a0064ef9a8d5f94a56f313c0" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 42513 } } peers { permanent_uuid: "02f9abe25a234b66a6509e002272b4e4" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 35891 } }
I20260619 02:43:32.227368 23620 catalog_manager.cc:5495] T c0579e78105340acb1d53f0d717fcd4d P 0fcd2bec719f400d88d330ed079857e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0fcd2bec719f400d88d330ed079857e6 (127.5.241.65). New cstate: current_term: 1 leader_uuid: "0fcd2bec719f400d88d330ed079857e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fcd2bec719f400d88d330ed079857e6" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 35855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9db1c960a0064ef9a8d5f94a56f313c0" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 42513 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02f9abe25a234b66a6509e002272b4e4" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 35891 } health_report { overall_health: UNKNOWN } } }
I20260619 02:43:32.243934 24081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:32.241436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58122 (local address 127.5.241.65:35855)
0619 02:43:32.241475 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:32.241477 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:32.241486 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:43:32.241571 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:32.241577 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:32.241596 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:32.241675 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:43:32.241783 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.242255 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.243102 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.243296 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.243400 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:32.243561 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:43:32.243666 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:43:32.243777 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:32.243798 (+ 21us) server_negotiation.cc:299] Negotiation successful
0619 02:43:32.243819 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 02:43:32.246165 24025 raft_consensus.cc:1261] T c0579e78105340acb1d53f0d717fcd4d P 9db1c960a0064ef9a8d5f94a56f313c0 [term 1 FOLLOWER]: Refusing update from remote peer 0fcd2bec719f400d88d330ed079857e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:43:32.246587 23894 raft_consensus.cc:1261] T c0579e78105340acb1d53f0d717fcd4d P 02f9abe25a234b66a6509e002272b4e4 [term 1 FOLLOWER]: Refusing update from remote peer 0fcd2bec719f400d88d330ed079857e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:43:32.246575 24093 consensus_queue.cc:1035] T c0579e78105340acb1d53f0d717fcd4d P 0fcd2bec719f400d88d330ed079857e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9db1c960a0064ef9a8d5f94a56f313c0" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 42513 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:43:32.246879 24093 consensus_queue.cc:1035] T c0579e78105340acb1d53f0d717fcd4d P 0fcd2bec719f400d88d330ed079857e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "02f9abe25a234b66a6509e002272b4e4" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 35891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 02:43:32.247324 24072 tablet.cc:2369] T c0579e78105340acb1d53f0d717fcd4d P 9db1c960a0064ef9a8d5f94a56f313c0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:43:32.248325 23810 tablet.cc:2369] T c0579e78105340acb1d53f0d717fcd4d P 0fcd2bec719f400d88d330ed079857e6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:43:32.250002 24099 mvcc.cc:205] Tried to move back new op lower bound from 7298404402156113920 to 7298404402080387072. Current Snapshot: MvccSnapshot[applied={T|T < 7298404402156113920}]
I20260619 02:43:32.251416 23621 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37122:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:43:32.251509 23621 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37122:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:43:32.252264 23621 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9fa66084103a46f0a12262b14a1fd2e2: Sending DeleteTablet for 3 replicas of tablet c0579e78105340acb1d53f0d717fcd4d
I20260619 02:43:32.252583 23743 tablet_service.cc:1514] Processing DeleteTablet for tablet c0579e78105340acb1d53f0d717fcd4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:43:32 UTC) from {username='slave'} at 127.0.0.1:58116
I20260619 02:43:32.252614 24005 tablet_service.cc:1514] Processing DeleteTablet for tablet c0579e78105340acb1d53f0d717fcd4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:43:32 UTC) from {username='slave'} at 127.0.0.1:56756
I20260619 02:43:32.252614 23874 tablet_service.cc:1514] Processing DeleteTablet for tablet c0579e78105340acb1d53f0d717fcd4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:43:32 UTC) from {username='slave'} at 127.0.0.1:38998
I20260619 02:43:32.252821 24108 tablet_replica.cc:330] T c0579e78105340acb1d53f0d717fcd4d P 02f9abe25a234b66a6509e002272b4e4: stopping tablet replica
I20260619 02:43:32.252919 24108 raft_consensus.cc:2229] T c0579e78105340acb1d53f0d717fcd4d P 02f9abe25a234b66a6509e002272b4e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:43:32.252952 24109 tablet_replica.cc:330] T c0579e78105340acb1d53f0d717fcd4d P 9db1c960a0064ef9a8d5f94a56f313c0: stopping tablet replica
I20260619 02:43:32.253022 24108 raft_consensus.cc:2258] T c0579e78105340acb1d53f0d717fcd4d P 02f9abe25a234b66a6509e002272b4e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:43:32.253122 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 23681
I20260619 02:43:32.253408 24108 ts_tablet_manager.cc:1905] T c0579e78105340acb1d53f0d717fcd4d P 02f9abe25a234b66a6509e002272b4e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:43:32.253896 24109 raft_consensus.cc:2229] T c0579e78105340acb1d53f0d717fcd4d P 9db1c960a0064ef9a8d5f94a56f313c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:43:32.254011 24109 raft_consensus.cc:2258] T c0579e78105340acb1d53f0d717fcd4d P 9db1c960a0064ef9a8d5f94a56f313c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:43:32.254410 24109 ts_tablet_manager.cc:1905] T c0579e78105340acb1d53f0d717fcd4d P 9db1c960a0064ef9a8d5f94a56f313c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:43:32.255285 24108 ts_tablet_manager.cc:1918] T c0579e78105340acb1d53f0d717fcd4d P 02f9abe25a234b66a6509e002272b4e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:43:32.255357 24108 log.cc:1196] T c0579e78105340acb1d53f0d717fcd4d P 02f9abe25a234b66a6509e002272b4e4: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/ts-1/wal/wals/c0579e78105340acb1d53f0d717fcd4d
I20260619 02:43:32.255643 24108 ts_tablet_manager.cc:1939] T c0579e78105340acb1d53f0d717fcd4d P 02f9abe25a234b66a6509e002272b4e4: Deleting consensus metadata
I20260619 02:43:32.256093 23607 catalog_manager.cc:4841] TS 02f9abe25a234b66a6509e002272b4e4 (127.5.241.66:35891): tablet c0579e78105340acb1d53f0d717fcd4d (table test-table [id=8734f30e4eac438a8172397adcab3ef0]) successfully deleted
I20260619 02:43:32.256289 24109 ts_tablet_manager.cc:1918] T c0579e78105340acb1d53f0d717fcd4d P 9db1c960a0064ef9a8d5f94a56f313c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:43:32.256357 24109 log.cc:1196] T c0579e78105340acb1d53f0d717fcd4d P 9db1c960a0064ef9a8d5f94a56f313c0: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836772852275-6085-0/minicluster-data/ts-2/wal/wals/c0579e78105340acb1d53f0d717fcd4d
I20260619 02:43:32.256587 24109 ts_tablet_manager.cc:1939] T c0579e78105340acb1d53f0d717fcd4d P 9db1c960a0064ef9a8d5f94a56f313c0: Deleting consensus metadata
I20260619 02:43:32.257121 23606 catalog_manager.cc:4841] TS 9db1c960a0064ef9a8d5f94a56f313c0 (127.5.241.67:42513): tablet c0579e78105340acb1d53f0d717fcd4d (table test-table [id=8734f30e4eac438a8172397adcab3ef0]) successfully deleted
W20260619 02:43:32.259423 23606 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.241.65:35855 (error 108)
I20260619 02:43:32.259545 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 23812
I20260619 02:43:32.260545 24080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:32.260410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:35855 (local address 127.0.0.1:58130)
0619 02:43:32.260469 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:32.260508 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.65:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260619 02:43:32.260692 23606 catalog_manager.cc:4584] TS 0fcd2bec719f400d88d330ed079857e6 (127.5.241.65:35855): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c0579e78105340acb1d53f0d717fcd4d: Network error: Client connection negotiation failed: client connection to 127.5.241.65:35855: connect: Connection refused (error 111)
I20260619 02:43:32.265100 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 23943
I20260619 02:43:32.269901 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 23590
2026-06-19T02:43:32Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (871 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:43:32 dist-test-slave-zpbg krb5kdc[24113](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:43:32 dist-test-slave-zpbg krb5kdc[24113](info): set up 2 sockets
Jun 19 02:43:32 dist-test-slave-zpbg krb5kdc[24113](info): commencing operation
krb5kdc: starting...
W20260619 02:43:33.310559 6085 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Jun 19 02:43:33 dist-test-slave-zpbg krb5kdc[24113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837013, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:43:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:43:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.241.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
I20260619 02:43:33.429591 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:35039
--webserver_interface=127.5.241.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:43509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:35039
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:43:33.509976 24126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:33.510181 24126 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:33.510203 24126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:33.511552 24126 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:43:33.511596 24126 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:33.511611 24126 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:33.511624 24126 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:43:33.511637 24126 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:43:33.513159 24126 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:43509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:35039
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:35039
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.241.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24126
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:43:33.513417 24126 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:33.513649 24126 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:33.518935 24134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:33.518967 24132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:33.519035 24126 server_base.cc:1034] running on GCE node
W20260619 02:43:33.518978 24131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:33.519567 24126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:33.519933 24126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:33.521116 24126 hybrid_clock.cc:648] HybridClock initialized: now 1781837013521097 us; error 30 us; skew 500 ppm
Jun 19 02:43:33 dist-test-slave-zpbg krb5kdc[24113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837013, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:33.523664 24126 init.cc:377] Logged in from keytab as kudu/127.5.241.126@KRBTEST.COM (short username kudu)
I20260619 02:43:33.524165 24126 webserver.cc:460] Webserver started at http://127.5.241.126:43803/ using document root <none> and password file <none>
I20260619 02:43:33.524394 24126 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:33.524464 24126 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:33.524636 24126 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:43:33.525722 24126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/master-0/data/instance:
uuid: "a9e1ee59daa14d17b406cceac0f3914e"
format_stamp: "Formatted at 2026-06-19 02:43:33 on dist-test-slave-zpbg"
server_key: "db4cf49affc25f53be83c7cc1f3b14a6"
server_key_iv: "5ae95debc2d6134b8a7000679db1dd39"
server_key_version: "encryptionkey@0"
I20260619 02:43:33.526103 24126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/master-0/wal/instance:
uuid: "a9e1ee59daa14d17b406cceac0f3914e"
format_stamp: "Formatted at 2026-06-19 02:43:33 on dist-test-slave-zpbg"
server_key: "db4cf49affc25f53be83c7cc1f3b14a6"
server_key_iv: "5ae95debc2d6134b8a7000679db1dd39"
server_key_version: "encryptionkey@0"
I20260619 02:43:33.527503 24126 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:43:33.528193 24141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:33.528374 24126 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:43:33.528473 24126 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "a9e1ee59daa14d17b406cceac0f3914e"
format_stamp: "Formatted at 2026-06-19 02:43:33 on dist-test-slave-zpbg"
server_key: "db4cf49affc25f53be83c7cc1f3b14a6"
server_key_iv: "5ae95debc2d6134b8a7000679db1dd39"
server_key_version: "encryptionkey@0"
I20260619 02:43:33.528555 24126 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:33.555466 24126 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:33.557669 24126 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:33.557852 24126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:33.561761 24126 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.126:35039
I20260619 02:43:33.561792 24193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.126:35039 every 8 connection(s)
I20260619 02:43:33.562176 24126 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:43:33.563182 24194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:33.564585 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 24126
I20260619 02:43:33.564684 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/master-0/wal/instance
I20260619 02:43:33.564924 6085 external_mini_cluster.cc:1428] Setting key f166deb0d5e8757994a9ede635113e8c
I20260619 02:43:33.565882 24194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9e1ee59daa14d17b406cceac0f3914e: Bootstrap starting.
I20260619 02:43:33.566584 24194 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9e1ee59daa14d17b406cceac0f3914e: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:33.566918 24194 log.cc:824] T 00000000000000000000000000000000 P a9e1ee59daa14d17b406cceac0f3914e: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:33.567608 24194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9e1ee59daa14d17b406cceac0f3914e: No bootstrap required, opened a new log
I20260619 02:43:33.569784 24194 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9e1ee59daa14d17b406cceac0f3914e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9e1ee59daa14d17b406cceac0f3914e" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 35039 } }
I20260619 02:43:33.569993 24194 raft_consensus.cc:374] T 00000000000000000000000000000000 P a9e1ee59daa14d17b406cceac0f3914e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:33.570034 24194 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9e1ee59daa14d17b406cceac0f3914e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9e1ee59daa14d17b406cceac0f3914e, State: Initialized, Role: FOLLOWER
I20260619 02:43:33.570245 24194 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9e1ee59daa14d17b406cceac0f3914e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9e1ee59daa14d17b406cceac0f3914e" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 35039 } }
I20260619 02:43:33.570345 24194 raft_consensus.cc:388] T 00000000000000000000000000000000 P a9e1ee59daa14d17b406cceac0f3914e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:43:33.570402 24194 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9e1ee59daa14d17b406cceac0f3914e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:43:33.570458 24194 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9e1ee59daa14d17b406cceac0f3914e [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:33.571090 24194 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9e1ee59daa14d17b406cceac0f3914e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9e1ee59daa14d17b406cceac0f3914e" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 35039 } }
I20260619 02:43:33.571213 24194 leader_election.cc:302] T 00000000000000000000000000000000 P a9e1ee59daa14d17b406cceac0f3914e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a9e1ee59daa14d17b406cceac0f3914e; no voters:
I20260619 02:43:33.571395 24194 leader_election.cc:288] T 00000000000000000000000000000000 P a9e1ee59daa14d17b406cceac0f3914e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:43:33.571563 24200 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9e1ee59daa14d17b406cceac0f3914e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:43:33.571646 24194 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9e1ee59daa14d17b406cceac0f3914e [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:43:33.571698 24200 raft_consensus.cc:686] T 00000000000000000000000000000000 P a9e1ee59daa14d17b406cceac0f3914e [term 1 LEADER]: Becoming Leader. State: Replica: a9e1ee59daa14d17b406cceac0f3914e, State: Running, Role: LEADER
I20260619 02:43:33.571910 24200 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9e1ee59daa14d17b406cceac0f3914e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9e1ee59daa14d17b406cceac0f3914e" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 35039 } }
Jun 19 02:43:33 dist-test-slave-zpbg krb5kdc[24113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837013, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:43:33.572310 24200 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9e1ee59daa14d17b406cceac0f3914e [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9e1ee59daa14d17b406cceac0f3914e. Latest consensus state: current_term: 1 leader_uuid: "a9e1ee59daa14d17b406cceac0f3914e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9e1ee59daa14d17b406cceac0f3914e" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 35039 } } }
I20260619 02:43:33.572389 24200 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9e1ee59daa14d17b406cceac0f3914e [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:33.572552 24201 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9e1ee59daa14d17b406cceac0f3914e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a9e1ee59daa14d17b406cceac0f3914e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9e1ee59daa14d17b406cceac0f3914e" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 35039 } } }
I20260619 02:43:33.572597 24208 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:43:33.572705 24201 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9e1ee59daa14d17b406cceac0f3914e [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:33.573310 24208 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:43:33.574958 24208 catalog_manager.cc:1347] Generated new cluster ID: 63dc7d2a496c4fa9a8eb706d784139d5
I20260619 02:43:33.575007 24208 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:43:33.576295 24197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:33.565470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46770 (local address 127.5.241.126:35039)
0619 02:43:33.565699 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:33.565708 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:33.565740 (+ 32us) server_negotiation.cc:407] Connection header received
0619 02:43:33.566297 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:33.566307 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:33.566379 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:33.566524 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:33.567311 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:33.568278 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:33.569384 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:33.569762 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:33.569839 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:33.572596 (+ 2757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:33.572600 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:33.572600 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:33.572612 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:33.574634 (+ 2022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:33.575133 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:33.575136 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:33.575136 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:33.575232 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:33.575448 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:33.575449 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:33.575449 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:33.575685 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:33.575827 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:33.575938 (+ 111us) server_negotiation.cc:299] Negotiation successful
0619 02:43:33.576074 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":124,"threads_started":1}
I20260619 02:43:33.581451 24208 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:43:33.582109 24208 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:43:33.586001 24208 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a9e1ee59daa14d17b406cceac0f3914e: Generated new TSK 0
I20260619 02:43:33.586164 24208 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.241.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab.
Entry for principal kudu/127.5.241.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab.
I20260619 02:43:33.616978 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.65:0
--local_ip_for_outbound_sockets=127.5.241.65
--webserver_interface=127.5.241.65
--webserver_port=0
--tserver_master_addrs=127.5.241.126:35039
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab
--principal=kudu/127.5.241.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:43509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:43:33.704480 24220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:33.704708 24220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:33.704746 24220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:33.705875 24220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:33.705940 24220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:33.705986 24220 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.65
I20260619 02:43:33.707489 24220 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:43509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab
--principal=kudu/127.5.241.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.241.65
--webserver_port=0
--tserver_master_addrs=127.5.241.126:35039
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24220
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.65
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:43:33.707732 24220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:33.707960 24220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:33.713083 24226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:33.713064 24225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:33.713064 24228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:33.713385 24220 server_base.cc:1034] running on GCE node
I20260619 02:43:33.713593 24220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:33.713837 24220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:33.714967 24220 hybrid_clock.cc:648] HybridClock initialized: now 1781837013714943 us; error 46 us; skew 500 ppm
Jun 19 02:43:33 dist-test-slave-zpbg krb5kdc[24113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837013, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:33.717607 24220 init.cc:377] Logged in from keytab as kudu/127.5.241.65@KRBTEST.COM (short username kudu)
I20260619 02:43:33.717980 24220 webserver.cc:460] Webserver started at http://127.5.241.65:33587/ using document root <none> and password file <none>
I20260619 02:43:33.718209 24220 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:33.718277 24220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:33.718431 24220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:43:33.719547 24220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/ts-0/data/instance:
uuid: "b610f71ffe96401d90128424ac66ff68"
format_stamp: "Formatted at 2026-06-19 02:43:33 on dist-test-slave-zpbg"
server_key: "8a7a3c2a0fe08881d7d5f9d2d196c43a"
server_key_iv: "aafd4b048a2be76431e93bd90c7522da"
server_key_version: "encryptionkey@0"
I20260619 02:43:33.719918 24220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/ts-0/wal/instance:
uuid: "b610f71ffe96401d90128424ac66ff68"
format_stamp: "Formatted at 2026-06-19 02:43:33 on dist-test-slave-zpbg"
server_key: "8a7a3c2a0fe08881d7d5f9d2d196c43a"
server_key_iv: "aafd4b048a2be76431e93bd90c7522da"
server_key_version: "encryptionkey@0"
I20260619 02:43:33.721256 24220 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:43:33.722074 24235 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:33.722292 24220 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:43:33.722409 24220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/ts-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/ts-0/wal
uuid: "b610f71ffe96401d90128424ac66ff68"
format_stamp: "Formatted at 2026-06-19 02:43:33 on dist-test-slave-zpbg"
server_key: "8a7a3c2a0fe08881d7d5f9d2d196c43a"
server_key_iv: "aafd4b048a2be76431e93bd90c7522da"
server_key_version: "encryptionkey@0"
I20260619 02:43:33.722483 24220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:33.732079 24220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:33.734354 24220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:33.734464 24220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:33.734637 24220 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:43:33.734982 24220 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:43:33.735018 24220 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:33.735039 24220 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:43:33.735052 24220 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:33.740875 24220 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.65:43713
I20260619 02:43:33.740921 24348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.65:43713 every 8 connection(s)
I20260619 02:43:33.741221 24220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
Jun 19 02:43:33 dist-test-slave-zpbg krb5kdc[24113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837013, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.65@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:43:33.749115 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 24220
I20260619 02:43:33.749243 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/ts-0/wal/instance
I20260619 02:43:33.749480 6085 external_mini_cluster.cc:1428] Setting key a050160025caa2abfdffd3f8fbbcee10
I20260619 02:43:33.750989 24197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:33.741909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:50873 (local address 127.5.241.126:35039)
0619 02:43:33.741992 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:33.741994 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:33.742172 (+ 178us) server_negotiation.cc:407] Connection header received
0619 02:43:33.742682 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:33.742683 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:33.742693 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:33.742753 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:33.743736 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:33.744239 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:33.745198 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:33.745396 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:33.745449 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:33.747704 (+ 2255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:33.747705 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:33.747706 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:33.747712 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:33.749146 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:33.749688 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:33.749689 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:33.749689 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:33.749774 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:33.750140 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:33.750141 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:33.750141 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:33.750303 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:33.750381 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:33.750848 (+ 467us) server_negotiation.cc:299] Negotiation successful
0619 02:43:33.750923 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 02:43:33.751364 24351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:33.741872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:35039 (local address 127.5.241.65:50873)
0619 02:43:33.742043 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:33.742087 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:33.742556 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:33.742872 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:33.742882 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:33.743458 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:43:33.743654 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:33.743659 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:33.744324 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:33.744325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:33.745124 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:33.745126 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:33.745485 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:33.745486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:33.745606 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:33.746069 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:43:33.746083 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:43:33.747600 (+ 1517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:43:33.749268 (+ 1668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:33.749275 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:33.749277 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:33.749590 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:33.749847 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:33.749848 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:33.749848 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:33.749974 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:33.750446 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:43:33.750454 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:43:33.750640 (+ 186us) client_negotiation.cc:769] Sending connection context
0619 02:43:33.750816 (+ 176us) client_negotiation.cc:240] Negotiation successful
0619 02:43:33.751016 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":96,"threads_started":1}
I20260619 02:43:33.751952 24349 heartbeater.cc:347] Connected to a master server at 127.5.241.126:35039
I20260619 02:43:33.752068 24349 heartbeater.cc:464] Registering TS with master...
I20260619 02:43:33.752262 24349 heartbeater.cc:511] Master 127.5.241.126:35039 requested a full tablet report, sending...
I20260619 02:43:33.752873 24158 ts_manager.cc:194] Registered new tserver with Master: b610f71ffe96401d90128424ac66ff68 (127.5.241.65:43713)
I20260619 02:43:33.753836 24158 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.65@KRBTEST.COM'} at 127.5.241.65:50873
WARNING: no policy specified for kudu/127.5.241.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab.
Entry for principal kudu/127.5.241.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab.
I20260619 02:43:33.778748 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.66:0
--local_ip_for_outbound_sockets=127.5.241.66
--webserver_interface=127.5.241.66
--webserver_port=0
--tserver_master_addrs=127.5.241.126:35039
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab
--principal=kudu/127.5.241.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:43509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:43:33.859937 24354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:33.860149 24354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:33.860172 24354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:33.861269 24354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:33.861310 24354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:33.861337 24354 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.66
I20260619 02:43:33.862706 24354 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:43509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab
--principal=kudu/127.5.241.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.241.66
--webserver_port=0
--tserver_master_addrs=127.5.241.126:35039
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24354
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.66
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:43:33.863049 24354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:33.863315 24354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:33.868623 24362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:33.868641 24359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:33.868631 24360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:33.868814 24354 server_base.cc:1034] running on GCE node
I20260619 02:43:33.868974 24354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:33.869159 24354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:33.870293 24354 hybrid_clock.cc:648] HybridClock initialized: now 1781837013870272 us; error 39 us; skew 500 ppm
Jun 19 02:43:33 dist-test-slave-zpbg krb5kdc[24113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837013, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:33.873052 24354 init.cc:377] Logged in from keytab as kudu/127.5.241.66@KRBTEST.COM (short username kudu)
I20260619 02:43:33.873553 24354 webserver.cc:460] Webserver started at http://127.5.241.66:33265/ using document root <none> and password file <none>
I20260619 02:43:33.873737 24354 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:33.873782 24354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:33.873869 24354 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:43:33.874938 24354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/ts-1/data/instance:
uuid: "4c5a1cedff02436e95d4571a14edf415"
format_stamp: "Formatted at 2026-06-19 02:43:33 on dist-test-slave-zpbg"
server_key: "a2dec62aad809c2e79050874ff084924"
server_key_iv: "246b7ea10f7aadb15b99099e89bd3d22"
server_key_version: "encryptionkey@0"
I20260619 02:43:33.875243 24354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/ts-1/wal/instance:
uuid: "4c5a1cedff02436e95d4571a14edf415"
format_stamp: "Formatted at 2026-06-19 02:43:33 on dist-test-slave-zpbg"
server_key: "a2dec62aad809c2e79050874ff084924"
server_key_iv: "246b7ea10f7aadb15b99099e89bd3d22"
server_key_version: "encryptionkey@0"
I20260619 02:43:33.876513 24354 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:43:33.877189 24369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:33.877357 24354 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:43:33.877447 24354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/ts-1/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/ts-1/wal
uuid: "4c5a1cedff02436e95d4571a14edf415"
format_stamp: "Formatted at 2026-06-19 02:43:33 on dist-test-slave-zpbg"
server_key: "a2dec62aad809c2e79050874ff084924"
server_key_iv: "246b7ea10f7aadb15b99099e89bd3d22"
server_key_version: "encryptionkey@0"
I20260619 02:43:33.877503 24354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:33.888266 24354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:33.890141 24354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:33.890254 24354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:33.890440 24354 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:43:33.890697 24354 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:43:33.890735 24354 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:33.890758 24354 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:43:33.890829 24354 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:33.896531 24354 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.66:45833
I20260619 02:43:33.896606 24482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.66:45833 every 8 connection(s)
I20260619 02:43:33.896929 24354 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
Jun 19 02:43:33 dist-test-slave-zpbg krb5kdc[24113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837013, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.66@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:43:33.903560 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 24354
I20260619 02:43:33.903674 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/ts-1/wal/instance
I20260619 02:43:33.903946 6085 external_mini_cluster.cc:1428] Setting key 88f4ec0087aab604532f225ed522630e
I20260619 02:43:33.906602 24197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:33.897651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:51995 (local address 127.5.241.126:35039)
0619 02:43:33.897721 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:33.897724 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:33.897949 (+ 225us) server_negotiation.cc:407] Connection header received
0619 02:43:33.898414 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:33.898415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:33.898424 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:33.898489 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:33.899500 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:33.900029 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:33.901005 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:33.901204 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:33.901266 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:33.903421 (+ 2155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:33.903423 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:33.903423 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:33.903432 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:33.905236 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:33.905605 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:33.905606 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:33.905607 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:33.905668 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:33.905897 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:33.905897 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:33.905898 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:33.906045 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:33.906120 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:33.906473 (+ 353us) server_negotiation.cc:299] Negotiation successful
0619 02:43:33.906539 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 02:43:33.906914 24485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:33.897618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:35039 (local address 127.5.241.66:51995)
0619 02:43:33.897816 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:33.897858 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:33.898334 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:33.898545 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:33.898549 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:33.899196 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:43:33.899390 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:33.899395 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:33.900120 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:33.900121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:33.900922 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:33.900924 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:33.901285 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:33.901286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:33.901389 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:33.901858 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:43:33.901867 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:43:33.903297 (+ 1430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:43:33.905313 (+ 2016us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:33.905318 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:33.905320 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:33.905532 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:33.905730 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:33.905731 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:33.905731 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:33.905834 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:33.906128 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:43:33.906134 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:43:33.906323 (+ 189us) client_negotiation.cc:769] Sending connection context
0619 02:43:33.906451 (+ 128us) client_negotiation.cc:240] Negotiation successful
0619 02:43:33.906632 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":114,"threads_started":1}
I20260619 02:43:33.907389 24483 heartbeater.cc:347] Connected to a master server at 127.5.241.126:35039
I20260619 02:43:33.907502 24483 heartbeater.cc:464] Registering TS with master...
I20260619 02:43:33.907677 24483 heartbeater.cc:511] Master 127.5.241.126:35039 requested a full tablet report, sending...
I20260619 02:43:33.908078 24158 ts_manager.cc:194] Registered new tserver with Master: 4c5a1cedff02436e95d4571a14edf415 (127.5.241.66:45833)
I20260619 02:43:33.908480 24158 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.66@KRBTEST.COM'} at 127.5.241.66:51995
WARNING: no policy specified for kudu/127.5.241.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab.
Entry for principal kudu/127.5.241.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab.
I20260619 02:43:33.931015 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.67:0
--local_ip_for_outbound_sockets=127.5.241.67
--webserver_interface=127.5.241.67
--webserver_port=0
--tserver_master_addrs=127.5.241.126:35039
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab
--principal=kudu/127.5.241.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:43509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:43:34.011847 24488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:34.012095 24488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:34.012135 24488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:34.013387 24488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:34.013460 24488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:34.013507 24488 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.67
I20260619 02:43:34.014988 24488 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:43509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab
--principal=kudu/127.5.241.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.241.67
--webserver_port=0
--tserver_master_addrs=127.5.241.126:35039
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24488
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.67
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:43:34.015246 24488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:34.015471 24488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:34.020759 24493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:34.020900 24496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:34.020759 24494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:34.021030 24488 server_base.cc:1034] running on GCE node
I20260619 02:43:34.021224 24488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:34.021417 24488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:34.022547 24488 hybrid_clock.cc:648] HybridClock initialized: now 1781837014022533 us; error 32 us; skew 500 ppm
Jun 19 02:43:34 dist-test-slave-zpbg krb5kdc[24113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837014, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:34.025193 24488 init.cc:377] Logged in from keytab as kudu/127.5.241.67@KRBTEST.COM (short username kudu)
I20260619 02:43:34.025739 24488 webserver.cc:460] Webserver started at http://127.5.241.67:45147/ using document root <none> and password file <none>
I20260619 02:43:34.025983 24488 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:34.026051 24488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:34.026190 24488 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:43:34.027326 24488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/ts-2/data/instance:
uuid: "b806f3d937f34500a3f2214c1bec4e66"
format_stamp: "Formatted at 2026-06-19 02:43:34 on dist-test-slave-zpbg"
server_key: "909126825d9fcbdb9b71900e8b2ced61"
server_key_iv: "6f06e7826ad6699c5375e64243dccbcd"
server_key_version: "encryptionkey@0"
I20260619 02:43:34.027690 24488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/ts-2/wal/instance:
uuid: "b806f3d937f34500a3f2214c1bec4e66"
format_stamp: "Formatted at 2026-06-19 02:43:34 on dist-test-slave-zpbg"
server_key: "909126825d9fcbdb9b71900e8b2ced61"
server_key_iv: "6f06e7826ad6699c5375e64243dccbcd"
server_key_version: "encryptionkey@0"
I20260619 02:43:34.029001 24488 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:43:34.029742 24503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:34.029923 24488 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:43:34.030021 24488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/ts-2/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/ts-2/wal
uuid: "b806f3d937f34500a3f2214c1bec4e66"
format_stamp: "Formatted at 2026-06-19 02:43:34 on dist-test-slave-zpbg"
server_key: "909126825d9fcbdb9b71900e8b2ced61"
server_key_iv: "6f06e7826ad6699c5375e64243dccbcd"
server_key_version: "encryptionkey@0"
I20260619 02:43:34.030108 24488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:34.043401 24488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:34.045675 24488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:34.045851 24488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:34.046083 24488 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:43:34.046396 24488 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:43:34.046450 24488 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:34.046490 24488 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:43:34.046517 24488 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:34.052237 24488 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.67:46107
I20260619 02:43:34.052291 24616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.67:46107 every 8 connection(s)
I20260619 02:43:34.052587 24488 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
I20260619 02:43:34.055889 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 24488
I20260619 02:43:34.056010 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/ts-2/wal/instance
I20260619 02:43:34.056200 6085 external_mini_cluster.cc:1428] Setting key babb0ca877b5e1f1b15bba24a106c74b
Jun 19 02:43:34 dist-test-slave-zpbg krb5kdc[24113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837014, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.67@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:43:34.062011 24197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:34.053294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:58793 (local address 127.5.241.126:35039)
0619 02:43:34.053369 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:34.053371 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:34.053544 (+ 173us) server_negotiation.cc:407] Connection header received
0619 02:43:34.053982 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:34.053983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:34.053994 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:34.054053 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:34.055079 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.055609 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.056997 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.057265 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.057339 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:34.059473 (+ 2134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:34.059475 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:34.059476 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:34.059484 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:34.060638 (+ 1154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:34.061039 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:34.061041 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:34.061041 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:34.061102 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:34.061357 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:34.061358 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:34.061358 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:34.061472 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:34.061551 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:34.061830 (+ 279us) server_negotiation.cc:299] Negotiation successful
0619 02:43:34.061918 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 02:43:34.062193 24619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:34.053225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:35039 (local address 127.5.241.67:58793)
0619 02:43:34.053392 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:34.053434 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:34.053898 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:34.054132 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:34.054135 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:34.054686 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:43:34.054952 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:34.054957 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.055766 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.055767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:34.056894 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:34.056897 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.057357 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.057358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:34.057489 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:34.057920 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:43:34.057928 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:43:34.059377 (+ 1449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:43:34.060734 (+ 1357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:34.060741 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:34.060743 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:34.060961 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:34.061171 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:34.061172 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:34.061172 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:34.061288 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:34.061541 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:43:34.061547 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:43:34.061686 (+ 139us) client_negotiation.cc:769] Sending connection context
0619 02:43:34.061808 (+ 122us) client_negotiation.cc:240] Negotiation successful
0619 02:43:34.061965 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":91,"threads_started":1}
I20260619 02:43:34.062682 24617 heartbeater.cc:347] Connected to a master server at 127.5.241.126:35039
I20260619 02:43:34.062762 24617 heartbeater.cc:464] Registering TS with master...
I20260619 02:43:34.062975 24617 heartbeater.cc:511] Master 127.5.241.126:35039 requested a full tablet report, sending...
I20260619 02:43:34.063349 24158 ts_manager.cc:194] Registered new tserver with Master: b806f3d937f34500a3f2214c1bec4e66 (127.5.241.67:46107)
I20260619 02:43:34.063737 24158 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.67@KRBTEST.COM'} at 127.5.241.67:58793
I20260619 02:43:34.067847 6085 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:43:34.075286 24197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:34.069174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46784 (local address 127.5.241.126:35039)
0619 02:43:34.069221 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:34.069223 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:34.069348 (+ 125us) server_negotiation.cc:407] Connection header received
0619 02:43:34.069374 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:34.069375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:34.069383 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:34.069431 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:34.069967 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.070414 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.071160 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.071382 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.071436 (+ 54us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:34.072756 (+ 1320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:34.072757 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:34.072757 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:34.072763 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:34.074119 (+ 1356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:34.074471 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:34.074472 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:34.074472 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:34.074520 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:34.074762 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:34.074762 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:34.074762 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:34.074931 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:34.075012 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:34.075124 (+ 112us) server_negotiation.cc:299] Negotiation successful
0619 02:43:34.075171 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 02:43:34.076844 24158 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46784:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:43:34.078377 24158 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:43:34.085160 24626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:34.080530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.67:46107 (local address 127.0.0.1:47474)
0619 02:43:34.080846 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:34.080889 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:34.080951 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:34.081475 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:34.081479 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:34.081481 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:43:34.081562 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:34.081564 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.082681 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.082682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:34.083966 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:34.083969 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.084849 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.084851 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:34.084931 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:34.084943 (+ 12us) client_negotiation.cc:769] Sending connection context
0619 02:43:34.085019 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 02:43:34.085074 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"mutex_wait_us":2,"thread_start_us":70,"threads_started":1}
I20260619 02:43:34.085206 24628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:34.080701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47474 (local address 127.5.241.67:46107)
0619 02:43:34.081290 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:34.081294 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:34.081304 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:43:34.081340 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:34.081342 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:34.081359 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:34.081439 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:43:34.081664 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.082573 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.084060 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.084759 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.084831 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:34.084888 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:34.085043 (+ 155us) server_negotiation.cc:299] Negotiation successful
0619 02:43:34.085089 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":100,"threads_started":1}
I20260619 02:43:34.085475 24629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:34.080759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.66:45833 (local address 127.0.0.1:42136)
0619 02:43:34.081841 (+ 1082us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:34.081851 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:34.081898 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:34.082074 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:34.082074 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:34.082076 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:43:34.082111 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:34.082113 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.083307 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.083308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:34.084283 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:34.084286 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.085245 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.085246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:34.085309 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:34.085312 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:43:34.085370 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 02:43:34.085411 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1070,"mutex_wait_us":21,"thread_start_us":46,"threads_started":1}
I20260619 02:43:34.085525 24630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:34.080899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42136 (local address 127.5.241.66:45833)
0619 02:43:34.081092 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:34.081102 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:34.081920 (+ 818us) server_negotiation.cc:407] Connection header received
0619 02:43:34.081963 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:34.081965 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:34.081981 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:34.082066 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:43:34.082169 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.083209 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.084375 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.085094 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.085160 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:34.085208 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:34.085403 (+ 195us) server_negotiation.cc:299] Negotiation successful
0619 02:43:34.085443 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":93,"threads_started":1}
I20260619 02:43:34.086066 24625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:34.080545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:43713 (local address 127.0.0.1:51952)
0619 02:43:34.080883 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:34.080894 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:34.080957 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:34.081693 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:34.081694 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:34.081697 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:43:34.081738 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:34.081739 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.083104 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.083105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:34.084223 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:34.084225 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.085006 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.085007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:34.085059 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:34.085063 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:43:34.085107 (+ 44us) client_negotiation.cc:240] Negotiation successful
0619 02:43:34.085140 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"mutex_wait_us":28,"thread_start_us":53,"threads_started":1}
I20260619 02:43:34.085984 24551 tablet_service.cc:1467] Processing CreateTablet for tablet 1b9aa1f028614361b18dad1b78b8aa9e (DEFAULT_TABLE table=test-table [id=9cb8f87e76334d008a469d34a4ccbdb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:43:34.086323 24627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:34.080676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51952 (local address 127.5.241.65:43713)
0619 02:43:34.081125 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:34.081128 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:34.081137 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:43:34.081192 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:34.081194 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:34.081533 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:34.081607 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:43:34.081826 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.082998 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.084310 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.084842 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.084914 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:34.084971 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:34.085201 (+ 230us) server_negotiation.cc:299] Negotiation successful
0619 02:43:34.086237 (+ 1036us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":74,"threads_started":1}
I20260619 02:43:34.086364 24551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b9aa1f028614361b18dad1b78b8aa9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:34.086411 24417 tablet_service.cc:1467] Processing CreateTablet for tablet 1b9aa1f028614361b18dad1b78b8aa9e (DEFAULT_TABLE table=test-table [id=9cb8f87e76334d008a469d34a4ccbdb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:43:34.086751 24417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b9aa1f028614361b18dad1b78b8aa9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:34.086925 24283 tablet_service.cc:1467] Processing CreateTablet for tablet 1b9aa1f028614361b18dad1b78b8aa9e (DEFAULT_TABLE table=test-table [id=9cb8f87e76334d008a469d34a4ccbdb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:43:34.087236 24283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b9aa1f028614361b18dad1b78b8aa9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:34.089483 24631 tablet_bootstrap.cc:492] T 1b9aa1f028614361b18dad1b78b8aa9e P 4c5a1cedff02436e95d4571a14edf415: Bootstrap starting.
I20260619 02:43:34.089885 24632 tablet_bootstrap.cc:492] T 1b9aa1f028614361b18dad1b78b8aa9e P b806f3d937f34500a3f2214c1bec4e66: Bootstrap starting.
I20260619 02:43:34.090178 24631 tablet_bootstrap.cc:654] T 1b9aa1f028614361b18dad1b78b8aa9e P 4c5a1cedff02436e95d4571a14edf415: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:34.090487 24633 tablet_bootstrap.cc:492] T 1b9aa1f028614361b18dad1b78b8aa9e P b610f71ffe96401d90128424ac66ff68: Bootstrap starting.
I20260619 02:43:34.090588 24632 tablet_bootstrap.cc:654] T 1b9aa1f028614361b18dad1b78b8aa9e P b806f3d937f34500a3f2214c1bec4e66: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:34.090762 24631 log.cc:824] T 1b9aa1f028614361b18dad1b78b8aa9e P 4c5a1cedff02436e95d4571a14edf415: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:34.091185 24633 tablet_bootstrap.cc:654] T 1b9aa1f028614361b18dad1b78b8aa9e P b610f71ffe96401d90128424ac66ff68: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:34.091223 24632 log.cc:824] T 1b9aa1f028614361b18dad1b78b8aa9e P b806f3d937f34500a3f2214c1bec4e66: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:34.091570 24631 tablet_bootstrap.cc:492] T 1b9aa1f028614361b18dad1b78b8aa9e P 4c5a1cedff02436e95d4571a14edf415: No bootstrap required, opened a new log
I20260619 02:43:34.091658 24631 ts_tablet_manager.cc:1397] T 1b9aa1f028614361b18dad1b78b8aa9e P 4c5a1cedff02436e95d4571a14edf415: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:43:34.091774 24633 log.cc:824] T 1b9aa1f028614361b18dad1b78b8aa9e P b610f71ffe96401d90128424ac66ff68: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:34.092103 24632 tablet_bootstrap.cc:492] T 1b9aa1f028614361b18dad1b78b8aa9e P b806f3d937f34500a3f2214c1bec4e66: No bootstrap required, opened a new log
I20260619 02:43:34.092175 24632 ts_tablet_manager.cc:1397] T 1b9aa1f028614361b18dad1b78b8aa9e P b806f3d937f34500a3f2214c1bec4e66: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:43:34.092482 24633 tablet_bootstrap.cc:492] T 1b9aa1f028614361b18dad1b78b8aa9e P b610f71ffe96401d90128424ac66ff68: No bootstrap required, opened a new log
I20260619 02:43:34.092564 24633 ts_tablet_manager.cc:1397] T 1b9aa1f028614361b18dad1b78b8aa9e P b610f71ffe96401d90128424ac66ff68: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260619 02:43:34.093665 24631 raft_consensus.cc:348] T 1b9aa1f028614361b18dad1b78b8aa9e P 4c5a1cedff02436e95d4571a14edf415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b806f3d937f34500a3f2214c1bec4e66" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 46107 } } peers { permanent_uuid: "b610f71ffe96401d90128424ac66ff68" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 43713 } } peers { permanent_uuid: "4c5a1cedff02436e95d4571a14edf415" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 45833 } }
I20260619 02:43:34.093879 24631 raft_consensus.cc:374] T 1b9aa1f028614361b18dad1b78b8aa9e P 4c5a1cedff02436e95d4571a14edf415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:34.093918 24631 raft_consensus.cc:729] T 1b9aa1f028614361b18dad1b78b8aa9e P 4c5a1cedff02436e95d4571a14edf415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c5a1cedff02436e95d4571a14edf415, State: Initialized, Role: FOLLOWER
I20260619 02:43:34.093907 24633 raft_consensus.cc:348] T 1b9aa1f028614361b18dad1b78b8aa9e P b610f71ffe96401d90128424ac66ff68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b806f3d937f34500a3f2214c1bec4e66" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 46107 } } peers { permanent_uuid: "b610f71ffe96401d90128424ac66ff68" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 43713 } } peers { permanent_uuid: "4c5a1cedff02436e95d4571a14edf415" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 45833 } }
I20260619 02:43:34.094065 24631 consensus_queue.cc:260] T 1b9aa1f028614361b18dad1b78b8aa9e P 4c5a1cedff02436e95d4571a14edf415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b806f3d937f34500a3f2214c1bec4e66" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 46107 } } peers { permanent_uuid: "b610f71ffe96401d90128424ac66ff68" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 43713 } } peers { permanent_uuid: "4c5a1cedff02436e95d4571a14edf415" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 45833 } }
I20260619 02:43:34.094111 24633 raft_consensus.cc:374] T 1b9aa1f028614361b18dad1b78b8aa9e P b610f71ffe96401d90128424ac66ff68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:34.094156 24633 raft_consensus.cc:729] T 1b9aa1f028614361b18dad1b78b8aa9e P b610f71ffe96401d90128424ac66ff68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b610f71ffe96401d90128424ac66ff68, State: Initialized, Role: FOLLOWER
I20260619 02:43:34.094285 24633 consensus_queue.cc:260] T 1b9aa1f028614361b18dad1b78b8aa9e P b610f71ffe96401d90128424ac66ff68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b806f3d937f34500a3f2214c1bec4e66" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 46107 } } peers { permanent_uuid: "b610f71ffe96401d90128424ac66ff68" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 43713 } } peers { permanent_uuid: "4c5a1cedff02436e95d4571a14edf415" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 45833 } }
I20260619 02:43:34.094334 24631 ts_tablet_manager.cc:1428] T 1b9aa1f028614361b18dad1b78b8aa9e P 4c5a1cedff02436e95d4571a14edf415: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 02:43:34.094228 24632 raft_consensus.cc:348] T 1b9aa1f028614361b18dad1b78b8aa9e P b806f3d937f34500a3f2214c1bec4e66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b806f3d937f34500a3f2214c1bec4e66" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 46107 } } peers { permanent_uuid: "b610f71ffe96401d90128424ac66ff68" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 43713 } } peers { permanent_uuid: "4c5a1cedff02436e95d4571a14edf415" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 45833 } }
I20260619 02:43:34.094389 24632 raft_consensus.cc:374] T 1b9aa1f028614361b18dad1b78b8aa9e P b806f3d937f34500a3f2214c1bec4e66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:34.094434 24632 raft_consensus.cc:729] T 1b9aa1f028614361b18dad1b78b8aa9e P b806f3d937f34500a3f2214c1bec4e66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b806f3d937f34500a3f2214c1bec4e66, State: Initialized, Role: FOLLOWER
I20260619 02:43:34.094447 24483 heartbeater.cc:503] Master 127.5.241.126:35039 was elected leader, sending a full tablet report...
I20260619 02:43:34.094548 24633 ts_tablet_manager.cc:1428] T 1b9aa1f028614361b18dad1b78b8aa9e P b610f71ffe96401d90128424ac66ff68: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 02:43:34.094547 24632 consensus_queue.cc:260] T 1b9aa1f028614361b18dad1b78b8aa9e P b806f3d937f34500a3f2214c1bec4e66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b806f3d937f34500a3f2214c1bec4e66" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 46107 } } peers { permanent_uuid: "b610f71ffe96401d90128424ac66ff68" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 43713 } } peers { permanent_uuid: "4c5a1cedff02436e95d4571a14edf415" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 45833 } }
I20260619 02:43:34.094667 24349 heartbeater.cc:503] Master 127.5.241.126:35039 was elected leader, sending a full tablet report...
I20260619 02:43:34.094848 24617 heartbeater.cc:503] Master 127.5.241.126:35039 was elected leader, sending a full tablet report...
I20260619 02:43:34.095468 24632 ts_tablet_manager.cc:1428] T 1b9aa1f028614361b18dad1b78b8aa9e P b806f3d937f34500a3f2214c1bec4e66: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260619 02:43:34.147588 24484 tablet.cc:2369] T 1b9aa1f028614361b18dad1b78b8aa9e P 4c5a1cedff02436e95d4571a14edf415: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:43:34.175374 24637 raft_consensus.cc:482] T 1b9aa1f028614361b18dad1b78b8aa9e P 4c5a1cedff02436e95d4571a14edf415 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:43:34.175537 24637 raft_consensus.cc:504] T 1b9aa1f028614361b18dad1b78b8aa9e P 4c5a1cedff02436e95d4571a14edf415 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b806f3d937f34500a3f2214c1bec4e66" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 46107 } } peers { permanent_uuid: "b610f71ffe96401d90128424ac66ff68" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 43713 } } peers { permanent_uuid: "4c5a1cedff02436e95d4571a14edf415" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 45833 } }
I20260619 02:43:34.175931 24637 leader_election.cc:288] T 1b9aa1f028614361b18dad1b78b8aa9e P 4c5a1cedff02436e95d4571a14edf415 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b806f3d937f34500a3f2214c1bec4e66 (127.5.241.67:46107), b610f71ffe96401d90128424ac66ff68 (127.5.241.65:43713)
I20260619 02:43:34.178808 24627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:34.176264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:41087 (local address 127.5.241.65:43713)
0619 02:43:34.176323 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:34.176325 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:34.176333 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:43:34.176366 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:34.176367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:34.176374 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:34.176435 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:43:34.176607 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.177092 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.178029 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.178417 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.178507 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:34.178516 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:34.178673 (+ 157us) server_negotiation.cc:299] Negotiation successful
0619 02:43:34.178710 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 02:43:34.178808 24485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:34.176126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:43713 (local address 127.5.241.66:41087)
0619 02:43:34.176191 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:34.176204 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:34.176264 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:34.176452 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:34.176453 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:34.176455 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:43:34.176532 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:34.176534 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.177179 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.177180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:34.177928 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:34.177930 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.178504 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.178505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:34.178589 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:34.178598 (+ 9us) client_negotiation.cc:769] Sending connection context
0619 02:43:34.178666 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 02:43:34.178715 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260619 02:43:34.178938 24628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:34.176236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:40381 (local address 127.5.241.67:46107)
0619 02:43:34.176298 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:34.176300 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:34.176454 (+ 154us) server_negotiation.cc:407] Connection header received
0619 02:43:34.176523 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:34.176524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:34.176534 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:34.176604 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:43:34.176781 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.177237 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.178171 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.178552 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.178638 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:34.178646 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:34.178835 (+ 189us) server_negotiation.cc:299] Negotiation successful
0619 02:43:34.178874 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 02:43:34.178951 24640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:34.176126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.67:46107 (local address 127.5.241.66:40381)
0619 02:43:34.176367 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:34.176376 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:34.176464 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:34.176619 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:34.176620 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:34.176622 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:43:34.176697 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:34.176699 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.177338 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.177339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:34.178075 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:34.178077 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.178650 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.178651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:34.178730 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:34.178734 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:43:34.178825 (+ 91us) client_negotiation.cc:240] Negotiation successful
0619 02:43:34.178867 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"mutex_wait_us":82,"thread_start_us":109,"threads_started":1}
I20260619 02:43:34.179174 24303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b9aa1f028614361b18dad1b78b8aa9e" candidate_uuid: "4c5a1cedff02436e95d4571a14edf415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b610f71ffe96401d90128424ac66ff68" is_pre_election: true
I20260619 02:43:34.179352 24303 raft_consensus.cc:2454] T 1b9aa1f028614361b18dad1b78b8aa9e P b610f71ffe96401d90128424ac66ff68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c5a1cedff02436e95d4571a14edf415 in term 0.
I20260619 02:43:34.179491 24571 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b9aa1f028614361b18dad1b78b8aa9e" candidate_uuid: "4c5a1cedff02436e95d4571a14edf415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b806f3d937f34500a3f2214c1bec4e66" is_pre_election: true
I20260619 02:43:34.179646 24571 raft_consensus.cc:2454] T 1b9aa1f028614361b18dad1b78b8aa9e P b806f3d937f34500a3f2214c1bec4e66 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c5a1cedff02436e95d4571a14edf415 in term 0.
I20260619 02:43:34.179633 24371 leader_election.cc:302] T 1b9aa1f028614361b18dad1b78b8aa9e P 4c5a1cedff02436e95d4571a14edf415 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c5a1cedff02436e95d4571a14edf415, b610f71ffe96401d90128424ac66ff68; no voters:
I20260619 02:43:34.179783 24637 raft_consensus.cc:2781] T 1b9aa1f028614361b18dad1b78b8aa9e P 4c5a1cedff02436e95d4571a14edf415 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:43:34.179868 24637 raft_consensus.cc:482] T 1b9aa1f028614361b18dad1b78b8aa9e P 4c5a1cedff02436e95d4571a14edf415 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:43:34.179894 24637 raft_consensus.cc:3037] T 1b9aa1f028614361b18dad1b78b8aa9e P 4c5a1cedff02436e95d4571a14edf415 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:34.180588 24637 raft_consensus.cc:504] T 1b9aa1f028614361b18dad1b78b8aa9e P 4c5a1cedff02436e95d4571a14edf415 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b806f3d937f34500a3f2214c1bec4e66" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 46107 } } peers { permanent_uuid: "b610f71ffe96401d90128424ac66ff68" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 43713 } } peers { permanent_uuid: "4c5a1cedff02436e95d4571a14edf415" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 45833 } }
I20260619 02:43:34.180732 24637 leader_election.cc:288] T 1b9aa1f028614361b18dad1b78b8aa9e P 4c5a1cedff02436e95d4571a14edf415 [CANDIDATE]: Term 1 election: Requested vote from peers b806f3d937f34500a3f2214c1bec4e66 (127.5.241.67:46107), b610f71ffe96401d90128424ac66ff68 (127.5.241.65:43713)
I20260619 02:43:34.180918 24571 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b9aa1f028614361b18dad1b78b8aa9e" candidate_uuid: "4c5a1cedff02436e95d4571a14edf415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b806f3d937f34500a3f2214c1bec4e66"
I20260619 02:43:34.180955 24303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b9aa1f028614361b18dad1b78b8aa9e" candidate_uuid: "4c5a1cedff02436e95d4571a14edf415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b610f71ffe96401d90128424ac66ff68"
I20260619 02:43:34.181026 24571 raft_consensus.cc:3037] T 1b9aa1f028614361b18dad1b78b8aa9e P b806f3d937f34500a3f2214c1bec4e66 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:34.181025 24303 raft_consensus.cc:3037] T 1b9aa1f028614361b18dad1b78b8aa9e P b610f71ffe96401d90128424ac66ff68 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:34.181726 24303 raft_consensus.cc:2454] T 1b9aa1f028614361b18dad1b78b8aa9e P b610f71ffe96401d90128424ac66ff68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c5a1cedff02436e95d4571a14edf415 in term 1.
I20260619 02:43:34.181860 24571 raft_consensus.cc:2454] T 1b9aa1f028614361b18dad1b78b8aa9e P b806f3d937f34500a3f2214c1bec4e66 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c5a1cedff02436e95d4571a14edf415 in term 1.
I20260619 02:43:34.181901 24371 leader_election.cc:302] T 1b9aa1f028614361b18dad1b78b8aa9e P 4c5a1cedff02436e95d4571a14edf415 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c5a1cedff02436e95d4571a14edf415, b610f71ffe96401d90128424ac66ff68; no voters:
I20260619 02:43:34.182005 24637 raft_consensus.cc:2781] T 1b9aa1f028614361b18dad1b78b8aa9e P 4c5a1cedff02436e95d4571a14edf415 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:43:34.182163 24637 raft_consensus.cc:686] T 1b9aa1f028614361b18dad1b78b8aa9e P 4c5a1cedff02436e95d4571a14edf415 [term 1 LEADER]: Becoming Leader. State: Replica: 4c5a1cedff02436e95d4571a14edf415, State: Running, Role: LEADER
I20260619 02:43:34.182387 24637 consensus_queue.cc:237] T 1b9aa1f028614361b18dad1b78b8aa9e P 4c5a1cedff02436e95d4571a14edf415 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b806f3d937f34500a3f2214c1bec4e66" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 46107 } } peers { permanent_uuid: "b610f71ffe96401d90128424ac66ff68" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 43713 } } peers { permanent_uuid: "4c5a1cedff02436e95d4571a14edf415" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 45833 } }
I20260619 02:43:34.183282 24156 catalog_manager.cc:5495] T 1b9aa1f028614361b18dad1b78b8aa9e P 4c5a1cedff02436e95d4571a14edf415 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c5a1cedff02436e95d4571a14edf415 (127.5.241.66). New cstate: current_term: 1 leader_uuid: "4c5a1cedff02436e95d4571a14edf415" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b806f3d937f34500a3f2214c1bec4e66" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 46107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b610f71ffe96401d90128424ac66ff68" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 43713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c5a1cedff02436e95d4571a14edf415" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 45833 } health_report { overall_health: HEALTHY } } }
I20260619 02:43:34.200055 24630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:34.197731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42152 (local address 127.5.241.66:45833)
0619 02:43:34.197793 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:34.197799 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:34.197808 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:43:34.197843 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:34.197844 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:34.197861 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:34.197920 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:43:34.198056 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.198519 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.199262 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.199447 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.199499 (+ 52us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:34.199659 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:43:34.199751 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:43:34.199825 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:34.199919 (+ 94us) server_negotiation.cc:299] Negotiation successful
0619 02:43:34.199933 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 02:43:34.201956 24571 raft_consensus.cc:1261] T 1b9aa1f028614361b18dad1b78b8aa9e P b806f3d937f34500a3f2214c1bec4e66 [term 1 FOLLOWER]: Refusing update from remote peer 4c5a1cedff02436e95d4571a14edf415: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:43:34.201956 24303 raft_consensus.cc:1261] T 1b9aa1f028614361b18dad1b78b8aa9e P b610f71ffe96401d90128424ac66ff68 [term 1 FOLLOWER]: Refusing update from remote peer 4c5a1cedff02436e95d4571a14edf415: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:43:34.202347 24641 consensus_queue.cc:1035] T 1b9aa1f028614361b18dad1b78b8aa9e P 4c5a1cedff02436e95d4571a14edf415 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b610f71ffe96401d90128424ac66ff68" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 43713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:43:34.202544 24637 consensus_queue.cc:1035] T 1b9aa1f028614361b18dad1b78b8aa9e P 4c5a1cedff02436e95d4571a14edf415 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b806f3d937f34500a3f2214c1bec4e66" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 46107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:43:34.205284 24645 mvcc.cc:205] Tried to move back new op lower bound from 7298404410168139776 to 7298404410091634688. Current Snapshot: MvccSnapshot[applied={T|T < 7298404410168139776}]
I20260619 02:43:34.207476 24156 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46784:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:43:34.207587 24156 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46784:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:43:34.208396 24156 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a9e1ee59daa14d17b406cceac0f3914e: Sending DeleteTablet for 3 replicas of tablet 1b9aa1f028614361b18dad1b78b8aa9e
I20260619 02:43:34.208729 24551 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b9aa1f028614361b18dad1b78b8aa9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:43:34 UTC) from {username='kudu', principal='kudu/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:47474
I20260619 02:43:34.208797 24283 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b9aa1f028614361b18dad1b78b8aa9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:43:34 UTC) from {username='kudu', principal='kudu/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:51952
I20260619 02:43:34.208945 24417 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b9aa1f028614361b18dad1b78b8aa9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:43:34 UTC) from {username='kudu', principal='kudu/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:42136
I20260619 02:43:34.209088 24655 tablet_replica.cc:330] T 1b9aa1f028614361b18dad1b78b8aa9e P b610f71ffe96401d90128424ac66ff68: stopping tablet replica
I20260619 02:43:34.209151 24654 tablet_replica.cc:330] T 1b9aa1f028614361b18dad1b78b8aa9e P b806f3d937f34500a3f2214c1bec4e66: stopping tablet replica
I20260619 02:43:34.209265 24654 raft_consensus.cc:2229] T 1b9aa1f028614361b18dad1b78b8aa9e P b806f3d937f34500a3f2214c1bec4e66 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:43:34.209297 24655 raft_consensus.cc:2229] T 1b9aa1f028614361b18dad1b78b8aa9e P b610f71ffe96401d90128424ac66ff68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:43:34.209375 24654 raft_consensus.cc:2258] T 1b9aa1f028614361b18dad1b78b8aa9e P b806f3d937f34500a3f2214c1bec4e66 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:43:34.209422 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 24220
I20260619 02:43:34.209517 24656 tablet_replica.cc:330] T 1b9aa1f028614361b18dad1b78b8aa9e P 4c5a1cedff02436e95d4571a14edf415: stopping tablet replica
I20260619 02:43:34.209602 24656 raft_consensus.cc:2229] T 1b9aa1f028614361b18dad1b78b8aa9e P 4c5a1cedff02436e95d4571a14edf415 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:43:34.209705 24656 raft_consensus.cc:2258] T 1b9aa1f028614361b18dad1b78b8aa9e P 4c5a1cedff02436e95d4571a14edf415 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:43:34.210227 24656 ts_tablet_manager.cc:1905] T 1b9aa1f028614361b18dad1b78b8aa9e P 4c5a1cedff02436e95d4571a14edf415: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:43:34.212059 24654 ts_tablet_manager.cc:1905] T 1b9aa1f028614361b18dad1b78b8aa9e P b806f3d937f34500a3f2214c1bec4e66: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:43:34.213620 24654 ts_tablet_manager.cc:1918] T 1b9aa1f028614361b18dad1b78b8aa9e P b806f3d937f34500a3f2214c1bec4e66: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:43:34.213701 24654 log.cc:1196] T 1b9aa1f028614361b18dad1b78b8aa9e P b806f3d937f34500a3f2214c1bec4e66: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/ts-2/wal/wals/1b9aa1f028614361b18dad1b78b8aa9e
I20260619 02:43:34.213989 24654 ts_tablet_manager.cc:1939] T 1b9aa1f028614361b18dad1b78b8aa9e P b806f3d937f34500a3f2214c1bec4e66: Deleting consensus metadata
I20260619 02:43:34.214409 24656 ts_tablet_manager.cc:1918] T 1b9aa1f028614361b18dad1b78b8aa9e P 4c5a1cedff02436e95d4571a14edf415: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:43:34.214478 24656 log.cc:1196] T 1b9aa1f028614361b18dad1b78b8aa9e P 4c5a1cedff02436e95d4571a14edf415: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836772852275-6085-0/minicluster-data/ts-1/wal/wals/1b9aa1f028614361b18dad1b78b8aa9e
I20260619 02:43:34.214437 24142 catalog_manager.cc:4841] TS b806f3d937f34500a3f2214c1bec4e66 (127.5.241.67:46107): tablet 1b9aa1f028614361b18dad1b78b8aa9e (table test-table [id=9cb8f87e76334d008a469d34a4ccbdb5]) successfully deleted
I20260619 02:43:34.214815 24656 ts_tablet_manager.cc:1939] T 1b9aa1f028614361b18dad1b78b8aa9e P 4c5a1cedff02436e95d4571a14edf415: Deleting consensus metadata
I20260619 02:43:34.215219 24142 catalog_manager.cc:4841] TS 4c5a1cedff02436e95d4571a14edf415 (127.5.241.66:45833): tablet 1b9aa1f028614361b18dad1b78b8aa9e (table test-table [id=9cb8f87e76334d008a469d34a4ccbdb5]) successfully deleted
W20260619 02:43:34.216054 24143 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.241.65:43713 (error 108)
I20260619 02:43:34.216336 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 24354
I20260619 02:43:34.216887 24626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:34.216385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:43713 (local address 127.0.0.1:51964)
0619 02:43:34.216800 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:34.216845 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.65:43713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393}
W20260619 02:43:34.217060 24143 catalog_manager.cc:4584] TS b610f71ffe96401d90128424ac66ff68 (127.5.241.65:43713): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1b9aa1f028614361b18dad1b78b8aa9e: Network error: Client connection negotiation failed: client connection to 127.5.241.65:43713: connect: Connection refused (error 111)
I20260619 02:43:34.221092 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 24488
I20260619 02:43:34.225605 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 24126
2026-06-19T02:43:34Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (1956 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-06-19T02:43:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:43:34Z Disabled control of system clock
I20260619 02:43:34.253060 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:42815
--webserver_interface=127.5.241.126
--webserver_port=0
--builtin_ntp_servers=127.5.241.84:36899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:42815
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 02:43:34.334127 24661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:34.334357 24661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:34.334379 24661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:34.335796 24661 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:43:34.335856 24661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:34.335880 24661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:34.335911 24661 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:43:34.335935 24661 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:43:34.337702 24661 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:36899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:42815
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:42815
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.241.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:43:34.337997 24661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:34.338397 24661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:34.343240 24669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:34.343245 24667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:34.343326 24666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:34.343346 24661 server_base.cc:1034] running on GCE node
I20260619 02:43:34.343786 24661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:34.344040 24661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:34.345211 24661 hybrid_clock.cc:648] HybridClock initialized: now 1781837014345205 us; error 46 us; skew 500 ppm
I20260619 02:43:34.346560 24661 webserver.cc:460] Webserver started at http://127.5.241.126:38649/ using document root <none> and password file <none>
I20260619 02:43:34.346884 24661 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:34.346949 24661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:34.347091 24661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:43:34.348253 24661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/master-0/data/instance:
uuid: "cb28d9a8f0824c7e9e06d0e7fdd8b5cb"
format_stamp: "Formatted at 2026-06-19 02:43:34 on dist-test-slave-zpbg"
server_key: "5fdb5a63744cd604f9c8e2de5f17cf3b"
server_key_iv: "b98fc60d994286817115749c225ceac0"
server_key_version: "encryptionkey@0"
I20260619 02:43:34.348624 24661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/master-0/wal/instance:
uuid: "cb28d9a8f0824c7e9e06d0e7fdd8b5cb"
format_stamp: "Formatted at 2026-06-19 02:43:34 on dist-test-slave-zpbg"
server_key: "5fdb5a63744cd604f9c8e2de5f17cf3b"
server_key_iv: "b98fc60d994286817115749c225ceac0"
server_key_version: "encryptionkey@0"
I20260619 02:43:34.349943 24661 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:43:34.350674 24675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:34.350931 24661 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:43:34.351035 24661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "cb28d9a8f0824c7e9e06d0e7fdd8b5cb"
format_stamp: "Formatted at 2026-06-19 02:43:34 on dist-test-slave-zpbg"
server_key: "5fdb5a63744cd604f9c8e2de5f17cf3b"
server_key_iv: "b98fc60d994286817115749c225ceac0"
server_key_version: "encryptionkey@0"
I20260619 02:43:34.351109 24661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:34.383514 24661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:34.383872 24661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:34.384032 24661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:34.387921 24661 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.126:42815
I20260619 02:43:34.387979 24727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.126:42815 every 8 connection(s)
I20260619 02:43:34.388316 24661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:43:34.389274 24728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:34.391642 24728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb28d9a8f0824c7e9e06d0e7fdd8b5cb: Bootstrap starting.
I20260619 02:43:34.392218 24728 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb28d9a8f0824c7e9e06d0e7fdd8b5cb: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:34.392513 24728 log.cc:824] T 00000000000000000000000000000000 P cb28d9a8f0824c7e9e06d0e7fdd8b5cb: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:34.393167 24728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb28d9a8f0824c7e9e06d0e7fdd8b5cb: No bootstrap required, opened a new log
I20260619 02:43:34.394726 24728 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb28d9a8f0824c7e9e06d0e7fdd8b5cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb28d9a8f0824c7e9e06d0e7fdd8b5cb" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 42815 } }
I20260619 02:43:34.394891 24728 raft_consensus.cc:374] T 00000000000000000000000000000000 P cb28d9a8f0824c7e9e06d0e7fdd8b5cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:34.394913 24728 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb28d9a8f0824c7e9e06d0e7fdd8b5cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb28d9a8f0824c7e9e06d0e7fdd8b5cb, State: Initialized, Role: FOLLOWER
I20260619 02:43:34.395063 24728 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb28d9a8f0824c7e9e06d0e7fdd8b5cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb28d9a8f0824c7e9e06d0e7fdd8b5cb" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 42815 } }
I20260619 02:43:34.395138 24728 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb28d9a8f0824c7e9e06d0e7fdd8b5cb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:43:34.395164 24728 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb28d9a8f0824c7e9e06d0e7fdd8b5cb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:43:34.395244 24728 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb28d9a8f0824c7e9e06d0e7fdd8b5cb [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:34.395754 24728 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb28d9a8f0824c7e9e06d0e7fdd8b5cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb28d9a8f0824c7e9e06d0e7fdd8b5cb" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 42815 } }
I20260619 02:43:34.395864 24728 leader_election.cc:302] T 00000000000000000000000000000000 P cb28d9a8f0824c7e9e06d0e7fdd8b5cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb28d9a8f0824c7e9e06d0e7fdd8b5cb; no voters:
I20260619 02:43:34.396044 24728 leader_election.cc:288] T 00000000000000000000000000000000 P cb28d9a8f0824c7e9e06d0e7fdd8b5cb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:43:34.396096 24731 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb28d9a8f0824c7e9e06d0e7fdd8b5cb [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:43:34.396309 24728 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb28d9a8f0824c7e9e06d0e7fdd8b5cb [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:43:34.396337 24731 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb28d9a8f0824c7e9e06d0e7fdd8b5cb [term 1 LEADER]: Becoming Leader. State: Replica: cb28d9a8f0824c7e9e06d0e7fdd8b5cb, State: Running, Role: LEADER
I20260619 02:43:34.396430 24731 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb28d9a8f0824c7e9e06d0e7fdd8b5cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb28d9a8f0824c7e9e06d0e7fdd8b5cb" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 42815 } }
I20260619 02:43:34.396875 24733 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb28d9a8f0824c7e9e06d0e7fdd8b5cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb28d9a8f0824c7e9e06d0e7fdd8b5cb. Latest consensus state: current_term: 1 leader_uuid: "cb28d9a8f0824c7e9e06d0e7fdd8b5cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb28d9a8f0824c7e9e06d0e7fdd8b5cb" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 42815 } } }
I20260619 02:43:34.396963 24733 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb28d9a8f0824c7e9e06d0e7fdd8b5cb [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:34.396970 24732 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb28d9a8f0824c7e9e06d0e7fdd8b5cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cb28d9a8f0824c7e9e06d0e7fdd8b5cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb28d9a8f0824c7e9e06d0e7fdd8b5cb" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 42815 } } }
I20260619 02:43:34.397044 24732 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb28d9a8f0824c7e9e06d0e7fdd8b5cb [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:34.397262 24740 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:43:34.397790 24740 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:43:34.397797 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 24661
I20260619 02:43:34.397892 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/master-0/wal/instance
I20260619 02:43:34.398135 6085 external_mini_cluster.cc:1428] Setting key 75f170495e66fc2ed3e2c8f4753de511
I20260619 02:43:34.399678 24740 catalog_manager.cc:1347] Generated new cluster ID: ad3724b764504b2db90b43038f963948
I20260619 02:43:34.399734 24740 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:43:34.404395 24750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:34.398611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37960 (local address 127.5.241.126:42815)
0619 02:43:34.398828 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:34.398837 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:34.398897 (+ 60us) server_negotiation.cc:407] Connection header received
0619 02:43:34.399026 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:34.399033 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:34.399122 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:34.399301 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:34.399525 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.400901 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.402038 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.402659 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.402797 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:34.403565 (+ 768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:34.403568 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:34.403569 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:43:34.403792 (+ 223us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:34.403835 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:43:34.403849 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:34.403921 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:34.404009 (+ 88us) server_negotiation.cc:299] Negotiation successful
0619 02:43:34.404086 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":74,"threads_started":1}
I20260619 02:43:34.404680 24740 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:43:34.405238 24740 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:43:34.408224 24740 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cb28d9a8f0824c7e9e06d0e7fdd8b5cb: Generated new TSK 0
I20260619 02:43:34.408399 24740 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:43:34.411720 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.65:0
--local_ip_for_outbound_sockets=127.5.241.65
--webserver_interface=127.5.241.65
--webserver_port=0
--tserver_master_addrs=127.5.241.126:42815
--builtin_ntp_servers=127.5.241.84:36899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 02:43:34.493359 24753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:34.493571 24753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:34.493597 24753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:34.494875 24753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:34.494915 24753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:34.494942 24753 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.65
I20260619 02:43:34.496460 24753 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:36899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.241.65
--webserver_port=0
--tserver_master_addrs=127.5.241.126:42815
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24753
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.65
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:43:34.496716 24753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:34.497141 24753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:34.502681 24759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:34.502728 24758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:34.502698 24761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:34.502956 24753 server_base.cc:1034] running on GCE node
I20260619 02:43:34.503249 24753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:34.503495 24753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:34.504699 24753 hybrid_clock.cc:648] HybridClock initialized: now 1781837014504660 us; error 47 us; skew 500 ppm
I20260619 02:43:34.506104 24753 webserver.cc:460] Webserver started at http://127.5.241.65:35809/ using document root <none> and password file <none>
I20260619 02:43:34.506368 24753 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:34.506439 24753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:34.506640 24753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:43:34.507932 24753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/ts-0/data/instance:
uuid: "acacf2b41c14469c8a450b400cd65333"
format_stamp: "Formatted at 2026-06-19 02:43:34 on dist-test-slave-zpbg"
server_key: "d30636f143cc6f5612d6d4957169aafc"
server_key_iv: "aa6644104ec941132ce26cfe75d9eebc"
server_key_version: "encryptionkey@0"
I20260619 02:43:34.508338 24753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/ts-0/wal/instance:
uuid: "acacf2b41c14469c8a450b400cd65333"
format_stamp: "Formatted at 2026-06-19 02:43:34 on dist-test-slave-zpbg"
server_key: "d30636f143cc6f5612d6d4957169aafc"
server_key_iv: "aa6644104ec941132ce26cfe75d9eebc"
server_key_version: "encryptionkey@0"
I20260619 02:43:34.509713 24753 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:43:34.510659 24767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:34.510831 24753 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:43:34.510943 24753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/ts-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/ts-0/wal
uuid: "acacf2b41c14469c8a450b400cd65333"
format_stamp: "Formatted at 2026-06-19 02:43:34 on dist-test-slave-zpbg"
server_key: "d30636f143cc6f5612d6d4957169aafc"
server_key_iv: "aa6644104ec941132ce26cfe75d9eebc"
server_key_version: "encryptionkey@0"
I20260619 02:43:34.511020 24753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:34.551072 24753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:34.551407 24753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:34.551571 24753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:34.551819 24753 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:43:34.552163 24753 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:43:34.552197 24753 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:34.552251 24753 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:43:34.552281 24753 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:34.558248 24753 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.65:36045
I20260619 02:43:34.558310 24880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.65:36045 every 8 connection(s)
I20260619 02:43:34.558678 24753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
I20260619 02:43:34.563758 24750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:34.559498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:60785 (local address 127.5.241.126:42815)
0619 02:43:34.559570 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:34.559572 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:34.559763 (+ 191us) server_negotiation.cc:407] Connection header received
0619 02:43:34.560223 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:34.560224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:34.560235 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:34.560303 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:34.560928 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.561403 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.562405 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.562585 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.562634 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:34.563560 (+ 926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:34.563561 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:34.563561 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:43:34.563565 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:34.563575 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:43:34.563578 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:34.563616 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:34.563674 (+ 58us) server_negotiation.cc:299] Negotiation successful
0619 02:43:34.563695 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 02:43:34.563994 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:34.559452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:42815 (local address 127.5.241.65:60785)
0619 02:43:34.559629 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:34.559669 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:34.560145 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:34.560356 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:34.560359 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:34.560366 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:43:34.560853 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:34.560857 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.561500 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.561501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:34.562325 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:34.562327 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.562655 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.562655 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:34.562841 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:34.563439 (+ 598us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:43:34.563439 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:43:34.563467 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:43:34.563490 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:43:34.563616 (+ 126us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:43:34.563618 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:43:34.563621 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:43:34.563666 (+ 45us) client_negotiation.cc:240] Negotiation successful
0619 02:43:34.563747 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":86,"threads_started":1}
I20260619 02:43:34.564586 24881 heartbeater.cc:347] Connected to a master server at 127.5.241.126:42815
I20260619 02:43:34.564670 24881 heartbeater.cc:464] Registering TS with master...
I20260619 02:43:34.564828 24881 heartbeater.cc:511] Master 127.5.241.126:42815 requested a full tablet report, sending...
I20260619 02:43:34.565281 24692 ts_manager.cc:194] Registered new tserver with Master: acacf2b41c14469c8a450b400cd65333 (127.5.241.65:36045)
I20260619 02:43:34.565897 24692 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.241.65:60785
I20260619 02:43:34.566942 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 24753
I20260619 02:43:34.567054 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/ts-0/wal/instance
I20260619 02:43:34.567283 6085 external_mini_cluster.cc:1428] Setting key f92c1cdb69e6457c38fcfebf5b4380d6
I20260619 02:43:34.568137 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.66:0
--local_ip_for_outbound_sockets=127.5.241.66
--webserver_interface=127.5.241.66
--webserver_port=0
--tserver_master_addrs=127.5.241.126:42815
--builtin_ntp_servers=127.5.241.84:36899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 02:43:34.649863 24884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:34.650067 24884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:34.650089 24884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:34.651336 24884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:34.651381 24884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:34.651407 24884 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.66
I20260619 02:43:34.652747 24884 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:36899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.241.66
--webserver_port=0
--tserver_master_addrs=127.5.241.126:42815
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24884
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.66
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:43:34.652992 24884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:34.653290 24884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:34.658624 24890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:34.658766 24884 server_base.cc:1034] running on GCE node
W20260619 02:43:34.658841 24892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:34.658900 24889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:34.659263 24884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:34.659521 24884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:34.660691 24884 hybrid_clock.cc:648] HybridClock initialized: now 1781837014660668 us; error 34 us; skew 500 ppm
I20260619 02:43:34.662081 24884 webserver.cc:460] Webserver started at http://127.5.241.66:35731/ using document root <none> and password file <none>
I20260619 02:43:34.662370 24884 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:34.662421 24884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:34.662516 24884 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:43:34.663653 24884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/ts-1/data/instance:
uuid: "8c50f55f75e24c12b576e81707dc182f"
format_stamp: "Formatted at 2026-06-19 02:43:34 on dist-test-slave-zpbg"
server_key: "5683c714f56c584161d4f845463248eb"
server_key_iv: "1e68192c875b4e5d8845a5417812fef4"
server_key_version: "encryptionkey@0"
I20260619 02:43:34.663947 24884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/ts-1/wal/instance:
uuid: "8c50f55f75e24c12b576e81707dc182f"
format_stamp: "Formatted at 2026-06-19 02:43:34 on dist-test-slave-zpbg"
server_key: "5683c714f56c584161d4f845463248eb"
server_key_iv: "1e68192c875b4e5d8845a5417812fef4"
server_key_version: "encryptionkey@0"
I20260619 02:43:34.665158 24884 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 02:43:34.665879 24898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:34.666061 24884 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:43:34.666131 24884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/ts-1/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/ts-1/wal
uuid: "8c50f55f75e24c12b576e81707dc182f"
format_stamp: "Formatted at 2026-06-19 02:43:34 on dist-test-slave-zpbg"
server_key: "5683c714f56c584161d4f845463248eb"
server_key_iv: "1e68192c875b4e5d8845a5417812fef4"
server_key_version: "encryptionkey@0"
I20260619 02:43:34.666182 24884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:34.678972 24884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:34.679240 24884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:34.679343 24884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:34.679517 24884 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:43:34.679864 24884 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:43:34.679900 24884 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:34.679960 24884 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:43:34.679989 24884 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:34.686049 24884 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.66:40687
I20260619 02:43:34.686112 25011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.66:40687 every 8 connection(s)
I20260619 02:43:34.686451 24884 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
I20260619 02:43:34.692083 24750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:34.687325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:47465 (local address 127.5.241.126:42815)
0619 02:43:34.687424 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:34.687426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:34.687568 (+ 142us) server_negotiation.cc:407] Connection header received
0619 02:43:34.688127 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:34.688129 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:34.688140 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:34.688201 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:34.688993 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.689492 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.690482 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.690712 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.690849 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:34.691737 (+ 888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:34.691739 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:34.691739 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:43:34.691744 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:34.691759 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:43:34.691763 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:34.691816 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:34.691983 (+ 167us) server_negotiation.cc:299] Negotiation successful
0619 02:43:34.692015 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 02:43:34.692229 25014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:34.687254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:42815 (local address 127.5.241.66:47465)
0619 02:43:34.687438 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:34.687473 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:34.687987 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:34.688253 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:34.688257 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:34.688265 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:43:34.688878 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:34.688885 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.689581 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.689582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:34.690378 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:34.690381 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.690847 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.690848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:34.690975 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:34.691511 (+ 536us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:43:34.691511 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:43:34.691541 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:43:34.691593 (+ 52us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:43:34.691861 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:43:34.691867 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:43:34.691870 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:43:34.691943 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 02:43:34.692029 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":70,"threads_started":1}
I20260619 02:43:34.692827 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 24884
I20260619 02:43:34.692941 25012 heartbeater.cc:347] Connected to a master server at 127.5.241.126:42815
I20260619 02:43:34.692981 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/ts-1/wal/instance
I20260619 02:43:34.693049 25012 heartbeater.cc:464] Registering TS with master...
I20260619 02:43:34.693222 6085 external_mini_cluster.cc:1428] Setting key 7ca9ed3edf46726b4bfed26f6c1862c1
I20260619 02:43:34.693264 25012 heartbeater.cc:511] Master 127.5.241.126:42815 requested a full tablet report, sending...
I20260619 02:43:34.693769 24692 ts_manager.cc:194] Registered new tserver with Master: 8c50f55f75e24c12b576e81707dc182f (127.5.241.66:40687)
I20260619 02:43:34.694260 24692 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.241.66:47465
I20260619 02:43:34.694554 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.67:0
--local_ip_for_outbound_sockets=127.5.241.67
--webserver_interface=127.5.241.67
--webserver_port=0
--tserver_master_addrs=127.5.241.126:42815
--builtin_ntp_servers=127.5.241.84:36899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 02:43:34.776136 25015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:34.776389 25015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:34.776429 25015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:34.777578 25015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:34.777648 25015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:34.777691 25015 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.67
I20260619 02:43:34.779122 25015 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:36899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.241.67
--webserver_port=0
--tserver_master_addrs=127.5.241.126:42815
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25015
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.67
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:43:34.779393 25015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:34.779687 25015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:34.784689 25023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:34.784806 25021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:34.784929 25015 server_base.cc:1034] running on GCE node
W20260619 02:43:34.784984 25020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:34.785218 25015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:34.785418 25015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:34.786597 25015 hybrid_clock.cc:648] HybridClock initialized: now 1781837014786591 us; error 38 us; skew 500 ppm
I20260619 02:43:34.787928 25015 webserver.cc:460] Webserver started at http://127.5.241.67:35237/ using document root <none> and password file <none>
I20260619 02:43:34.788129 25015 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:34.788172 25015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:34.788271 25015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:43:34.789371 25015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/ts-2/data/instance:
uuid: "94dc4142e39d4affa12e736cb889ab46"
format_stamp: "Formatted at 2026-06-19 02:43:34 on dist-test-slave-zpbg"
server_key: "1c7be373cc7b7ea025dc21a2a2fe0f4c"
server_key_iv: "a83e2e9e48193af587696aa9383922af"
server_key_version: "encryptionkey@0"
I20260619 02:43:34.789671 25015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/ts-2/wal/instance:
uuid: "94dc4142e39d4affa12e736cb889ab46"
format_stamp: "Formatted at 2026-06-19 02:43:34 on dist-test-slave-zpbg"
server_key: "1c7be373cc7b7ea025dc21a2a2fe0f4c"
server_key_iv: "a83e2e9e48193af587696aa9383922af"
server_key_version: "encryptionkey@0"
I20260619 02:43:34.791220 25015 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:43:34.791951 25029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:34.792172 25015 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 02:43:34.792258 25015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/ts-2/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/ts-2/wal
uuid: "94dc4142e39d4affa12e736cb889ab46"
format_stamp: "Formatted at 2026-06-19 02:43:34 on dist-test-slave-zpbg"
server_key: "1c7be373cc7b7ea025dc21a2a2fe0f4c"
server_key_iv: "a83e2e9e48193af587696aa9383922af"
server_key_version: "encryptionkey@0"
I20260619 02:43:34.792322 25015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:34.819454 25015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:34.819728 25015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:34.819831 25015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:34.820006 25015 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:43:34.820351 25015 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:43:34.820391 25015 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:34.820415 25015 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:43:34.820462 25015 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:34.826339 25015 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.67:33109
I20260619 02:43:34.826416 25142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.67:33109 every 8 connection(s)
I20260619 02:43:34.826761 25015 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
I20260619 02:43:34.829650 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 25015
I20260619 02:43:34.829794 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/ts-2/wal/instance
I20260619 02:43:34.830018 6085 external_mini_cluster.cc:1428] Setting key 3651c959e651548a0ff60b8888d42566
I20260619 02:43:34.832662 24750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:34.827606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:35117 (local address 127.5.241.126:42815)
0619 02:43:34.827666 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:34.827669 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:34.827862 (+ 193us) server_negotiation.cc:407] Connection header received
0619 02:43:34.828368 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:34.828369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:34.828381 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:34.828447 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:34.829126 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.829666 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.831255 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.831447 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.831509 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:34.832418 (+ 909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:34.832420 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:34.832420 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:43:34.832424 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:34.832438 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:43:34.832442 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:34.832489 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:34.832579 (+ 90us) server_negotiation.cc:299] Negotiation successful
0619 02:43:34.832605 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 02:43:34.832896 25145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:34.827536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:42815 (local address 127.5.241.67:35117)
0619 02:43:34.827731 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:34.827771 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:34.828250 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:34.828523 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:34.828527 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:34.828535 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:43:34.829046 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:34.829049 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.829790 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.829791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:34.831127 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:34.831130 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.831547 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.831548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:34.831680 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:34.832265 (+ 585us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:43:34.832266 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:43:34.832294 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:43:34.832317 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:43:34.832502 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:43:34.832506 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:43:34.832509 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:43:34.832561 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 02:43:34.832666 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":94,"threads_started":1}
I20260619 02:43:34.833393 25143 heartbeater.cc:347] Connected to a master server at 127.5.241.126:42815
I20260619 02:43:34.833472 25143 heartbeater.cc:464] Registering TS with master...
I20260619 02:43:34.833671 25143 heartbeater.cc:511] Master 127.5.241.126:42815 requested a full tablet report, sending...
I20260619 02:43:34.834048 24692 ts_manager.cc:194] Registered new tserver with Master: 94dc4142e39d4affa12e736cb889ab46 (127.5.241.67:33109)
I20260619 02:43:34.834648 24692 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.241.67:35117
I20260619 02:43:34.841835 6085 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:43:34.846107 24750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:34.843236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37968 (local address 127.5.241.126:42815)
0619 02:43:34.843317 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:34.843319 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:34.843431 (+ 112us) server_negotiation.cc:407] Connection header received
0619 02:43:34.843470 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:34.843470 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:34.843481 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:34.843530 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:34.843679 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.844136 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.844841 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.845089 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.845144 (+ 55us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:34.845836 (+ 692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:34.845837 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:34.845837 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:43:34.845841 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:34.845852 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:43:34.845855 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:34.845918 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:34.846030 (+ 112us) server_negotiation.cc:299] Negotiation successful
0619 02:43:34.846041 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 02:43:34.847888 24692 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37968:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:43:34.849445 24692 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:43:34.856235 25151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:34.851984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.66:40687 (local address 127.0.0.1:42746)
0619 02:43:34.852289 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:34.852300 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:34.852369 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:34.852601 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:34.852604 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:34.852607 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:43:34.852689 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:34.852692 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.853783 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.853784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:34.855007 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:34.855010 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.855924 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.855925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:34.856008 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:34.856024 (+ 16us) client_negotiation.cc:769] Sending connection context
0619 02:43:34.856096 (+ 72us) client_negotiation.cc:240] Negotiation successful
0619 02:43:34.856150 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":76,"threads_started":1}
I20260619 02:43:34.856405 25155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:34.852159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.67:33109 (local address 127.0.0.1:55228)
0619 02:43:34.852534 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:34.852545 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:34.852600 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:34.852827 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:34.852828 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:34.852830 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:43:34.852874 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:34.852875 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.854209 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.854209 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:34.855327 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:34.855330 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.856165 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.856166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:34.856225 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:34.856227 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 02:43:34.856288 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 02:43:34.856330 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":2,"thread_start_us":64,"threads_started":1}
I20260619 02:43:34.856572 25152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:34.851926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:36045 (local address 127.0.0.1:54420)
0619 02:43:34.852125 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:34.852167 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:34.852232 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:34.852877 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:34.852877 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:34.852879 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:43:34.852921 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:34.852922 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.854202 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.854203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:34.855362 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:34.855364 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.856343 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.856344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:34.856416 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:34.856419 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:43:34.856466 (+ 47us) client_negotiation.cc:240] Negotiation successful
0619 02:43:34.856502 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":21,"thread_start_us":54,"threads_started":1}
I20260619 02:43:34.856261 25153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:34.852071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42746 (local address 127.5.241.66:40687)
0619 02:43:34.852298 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:34.852304 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:34.852356 (+ 52us) server_negotiation.cc:407] Connection header received
0619 02:43:34.852443 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:34.852445 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:34.852461 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:34.852783 (+ 322us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:43:34.852865 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.853667 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.855096 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.855829 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.855903 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:34.855992 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:34.856126 (+ 134us) server_negotiation.cc:299] Negotiation successful
0619 02:43:34.856169 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":105,"threads_started":1}
I20260619 02:43:34.856647 25156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:34.852356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55228 (local address 127.5.241.67:33109)
0619 02:43:34.852587 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:34.852591 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:34.852602 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:43:34.852670 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:34.852672 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:34.852687 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:34.852761 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:43:34.852949 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.854125 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.855421 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.855979 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.856046 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:34.856133 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:34.856505 (+ 372us) server_negotiation.cc:299] Negotiation successful
0619 02:43:34.856558 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":85,"threads_started":1}
I20260619 02:43:34.856853 25154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:34.852109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54420 (local address 127.5.241.65:36045)
0619 02:43:34.852726 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:34.852729 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:34.852739 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:43:34.852774 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:34.852776 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:34.852789 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:34.852863 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:43:34.852983 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.854103 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.855454 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.856158 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.856225 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:34.856304 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:34.856730 (+ 426us) server_negotiation.cc:299] Negotiation successful
0619 02:43:34.856772 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":90,"threads_started":1}
I20260619 02:43:34.857278 24946 tablet_service.cc:1467] Processing CreateTablet for tablet 29ea8f6fab274720895167859ba05f78 (DEFAULT_TABLE table=test-table [id=9381fe7ebabe4827956ed0a1a270fb14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:43:34.857426 25077 tablet_service.cc:1467] Processing CreateTablet for tablet 29ea8f6fab274720895167859ba05f78 (DEFAULT_TABLE table=test-table [id=9381fe7ebabe4827956ed0a1a270fb14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:43:34.857426 24815 tablet_service.cc:1467] Processing CreateTablet for tablet 29ea8f6fab274720895167859ba05f78 (DEFAULT_TABLE table=test-table [id=9381fe7ebabe4827956ed0a1a270fb14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:43:34.857650 24946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29ea8f6fab274720895167859ba05f78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:34.857757 25077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29ea8f6fab274720895167859ba05f78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:34.857757 24815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29ea8f6fab274720895167859ba05f78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:34.860265 25157 tablet_bootstrap.cc:492] T 29ea8f6fab274720895167859ba05f78 P 8c50f55f75e24c12b576e81707dc182f: Bootstrap starting.
I20260619 02:43:34.860951 25157 tablet_bootstrap.cc:654] T 29ea8f6fab274720895167859ba05f78 P 8c50f55f75e24c12b576e81707dc182f: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:34.861050 25159 tablet_bootstrap.cc:492] T 29ea8f6fab274720895167859ba05f78 P acacf2b41c14469c8a450b400cd65333: Bootstrap starting.
I20260619 02:43:34.861364 25157 log.cc:824] T 29ea8f6fab274720895167859ba05f78 P 8c50f55f75e24c12b576e81707dc182f: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:34.861698 25159 tablet_bootstrap.cc:654] T 29ea8f6fab274720895167859ba05f78 P acacf2b41c14469c8a450b400cd65333: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:34.861934 25159 log.cc:824] T 29ea8f6fab274720895167859ba05f78 P acacf2b41c14469c8a450b400cd65333: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:34.862177 25158 tablet_bootstrap.cc:492] T 29ea8f6fab274720895167859ba05f78 P 94dc4142e39d4affa12e736cb889ab46: Bootstrap starting.
I20260619 02:43:34.862708 25159 tablet_bootstrap.cc:492] T 29ea8f6fab274720895167859ba05f78 P acacf2b41c14469c8a450b400cd65333: No bootstrap required, opened a new log
I20260619 02:43:34.862716 25158 tablet_bootstrap.cc:654] T 29ea8f6fab274720895167859ba05f78 P 94dc4142e39d4affa12e736cb889ab46: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:34.862845 25159 ts_tablet_manager.cc:1397] T 29ea8f6fab274720895167859ba05f78 P acacf2b41c14469c8a450b400cd65333: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:43:34.863065 25158 log.cc:824] T 29ea8f6fab274720895167859ba05f78 P 94dc4142e39d4affa12e736cb889ab46: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:34.863945 25157 tablet_bootstrap.cc:492] T 29ea8f6fab274720895167859ba05f78 P 8c50f55f75e24c12b576e81707dc182f: No bootstrap required, opened a new log
I20260619 02:43:34.864020 25157 ts_tablet_manager.cc:1397] T 29ea8f6fab274720895167859ba05f78 P 8c50f55f75e24c12b576e81707dc182f: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260619 02:43:34.865700 25159 raft_consensus.cc:348] T 29ea8f6fab274720895167859ba05f78 P acacf2b41c14469c8a450b400cd65333 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acacf2b41c14469c8a450b400cd65333" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 36045 } } peers { permanent_uuid: "8c50f55f75e24c12b576e81707dc182f" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 40687 } } peers { permanent_uuid: "94dc4142e39d4affa12e736cb889ab46" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 33109 } }
I20260619 02:43:34.865880 25159 raft_consensus.cc:374] T 29ea8f6fab274720895167859ba05f78 P acacf2b41c14469c8a450b400cd65333 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:34.865921 25159 raft_consensus.cc:729] T 29ea8f6fab274720895167859ba05f78 P acacf2b41c14469c8a450b400cd65333 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acacf2b41c14469c8a450b400cd65333, State: Initialized, Role: FOLLOWER
I20260619 02:43:34.866055 25159 consensus_queue.cc:260] T 29ea8f6fab274720895167859ba05f78 P acacf2b41c14469c8a450b400cd65333 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acacf2b41c14469c8a450b400cd65333" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 36045 } } peers { permanent_uuid: "8c50f55f75e24c12b576e81707dc182f" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 40687 } } peers { permanent_uuid: "94dc4142e39d4affa12e736cb889ab46" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 33109 } }
I20260619 02:43:34.866334 25159 ts_tablet_manager.cc:1428] T 29ea8f6fab274720895167859ba05f78 P acacf2b41c14469c8a450b400cd65333: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 02:43:34.866386 24881 heartbeater.cc:503] Master 127.5.241.126:42815 was elected leader, sending a full tablet report...
I20260619 02:43:34.866870 25157 raft_consensus.cc:348] T 29ea8f6fab274720895167859ba05f78 P 8c50f55f75e24c12b576e81707dc182f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acacf2b41c14469c8a450b400cd65333" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 36045 } } peers { permanent_uuid: "8c50f55f75e24c12b576e81707dc182f" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 40687 } } peers { permanent_uuid: "94dc4142e39d4affa12e736cb889ab46" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 33109 } }
I20260619 02:43:34.867043 25157 raft_consensus.cc:374] T 29ea8f6fab274720895167859ba05f78 P 8c50f55f75e24c12b576e81707dc182f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:34.867086 25157 raft_consensus.cc:729] T 29ea8f6fab274720895167859ba05f78 P 8c50f55f75e24c12b576e81707dc182f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c50f55f75e24c12b576e81707dc182f, State: Initialized, Role: FOLLOWER
I20260619 02:43:34.867228 25157 consensus_queue.cc:260] T 29ea8f6fab274720895167859ba05f78 P 8c50f55f75e24c12b576e81707dc182f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acacf2b41c14469c8a450b400cd65333" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 36045 } } peers { permanent_uuid: "8c50f55f75e24c12b576e81707dc182f" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 40687 } } peers { permanent_uuid: "94dc4142e39d4affa12e736cb889ab46" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 33109 } }
I20260619 02:43:34.867288 25158 tablet_bootstrap.cc:492] T 29ea8f6fab274720895167859ba05f78 P 94dc4142e39d4affa12e736cb889ab46: No bootstrap required, opened a new log
I20260619 02:43:34.867436 25158 ts_tablet_manager.cc:1397] T 29ea8f6fab274720895167859ba05f78 P 94dc4142e39d4affa12e736cb889ab46: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260619 02:43:34.867514 25157 ts_tablet_manager.cc:1428] T 29ea8f6fab274720895167859ba05f78 P 8c50f55f75e24c12b576e81707dc182f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 02:43:34.867630 25012 heartbeater.cc:503] Master 127.5.241.126:42815 was elected leader, sending a full tablet report...
I20260619 02:43:34.869645 25158 raft_consensus.cc:348] T 29ea8f6fab274720895167859ba05f78 P 94dc4142e39d4affa12e736cb889ab46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acacf2b41c14469c8a450b400cd65333" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 36045 } } peers { permanent_uuid: "8c50f55f75e24c12b576e81707dc182f" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 40687 } } peers { permanent_uuid: "94dc4142e39d4affa12e736cb889ab46" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 33109 } }
I20260619 02:43:34.869805 25158 raft_consensus.cc:374] T 29ea8f6fab274720895167859ba05f78 P 94dc4142e39d4affa12e736cb889ab46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:34.869841 25158 raft_consensus.cc:729] T 29ea8f6fab274720895167859ba05f78 P 94dc4142e39d4affa12e736cb889ab46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94dc4142e39d4affa12e736cb889ab46, State: Initialized, Role: FOLLOWER
I20260619 02:43:34.869974 25158 consensus_queue.cc:260] T 29ea8f6fab274720895167859ba05f78 P 94dc4142e39d4affa12e736cb889ab46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acacf2b41c14469c8a450b400cd65333" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 36045 } } peers { permanent_uuid: "8c50f55f75e24c12b576e81707dc182f" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 40687 } } peers { permanent_uuid: "94dc4142e39d4affa12e736cb889ab46" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 33109 } }
I20260619 02:43:34.870221 25158 ts_tablet_manager.cc:1428] T 29ea8f6fab274720895167859ba05f78 P 94dc4142e39d4affa12e736cb889ab46: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 02:43:34.870283 25143 heartbeater.cc:503] Master 127.5.241.126:42815 was elected leader, sending a full tablet report...
I20260619 02:43:34.931492 25165 raft_consensus.cc:482] T 29ea8f6fab274720895167859ba05f78 P 94dc4142e39d4affa12e736cb889ab46 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:43:34.931615 25165 raft_consensus.cc:504] T 29ea8f6fab274720895167859ba05f78 P 94dc4142e39d4affa12e736cb889ab46 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acacf2b41c14469c8a450b400cd65333" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 36045 } } peers { permanent_uuid: "8c50f55f75e24c12b576e81707dc182f" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 40687 } } peers { permanent_uuid: "94dc4142e39d4affa12e736cb889ab46" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 33109 } }
I20260619 02:43:34.931985 25165 leader_election.cc:288] T 29ea8f6fab274720895167859ba05f78 P 94dc4142e39d4affa12e736cb889ab46 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acacf2b41c14469c8a450b400cd65333 (127.5.241.65:36045), 8c50f55f75e24c12b576e81707dc182f (127.5.241.66:40687)
I20260619 02:43:34.934695 25153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:34.932227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:59089 (local address 127.5.241.66:40687)
0619 02:43:34.932273 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:34.932275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:34.932285 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:43:34.932336 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:34.932337 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:34.932349 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:34.932428 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:43:34.932582 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.933053 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.933986 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.934343 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.934425 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:34.934433 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:34.934581 (+ 148us) server_negotiation.cc:299] Negotiation successful
0619 02:43:34.934618 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 02:43:34.934687 25145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:34.932151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.66:40687 (local address 127.5.241.67:59089)
0619 02:43:34.932207 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:34.932219 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:34.932260 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:34.932432 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:34.932434 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:34.932435 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:43:34.932506 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:34.932507 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.933149 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.933150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:34.933883 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:34.933885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.934433 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.934434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:34.934505 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:34.934508 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:43:34.934575 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 02:43:34.934615 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260619 02:43:34.934952 25154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:34.932239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:44979 (local address 127.5.241.65:36045)
0619 02:43:34.932325 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:34.932327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:34.932438 (+ 111us) server_negotiation.cc:407] Connection header received
0619 02:43:34.932500 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:34.932501 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:34.932516 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:34.932586 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:43:34.932738 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.933188 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.934144 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.934510 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.934610 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:34.934619 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:34.934814 (+ 195us) server_negotiation.cc:299] Negotiation successful
0619 02:43:34.934868 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 02:43:34.935034 24966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29ea8f6fab274720895167859ba05f78" candidate_uuid: "94dc4142e39d4affa12e736cb889ab46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c50f55f75e24c12b576e81707dc182f" is_pre_election: true
I20260619 02:43:34.935199 25166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:34.932152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:36045 (local address 127.5.241.67:44979)
0619 02:43:34.932361 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:34.932370 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:34.932443 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:34.932591 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:34.932591 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:34.932593 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:43:34.932666 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:34.932668 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.933314 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.933315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:34.934047 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:34.934049 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.934626 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.934627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:34.934696 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:34.934699 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:43:34.935062 (+ 363us) client_negotiation.cc:240] Negotiation successful
0619 02:43:34.935108 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":15,"thread_start_us":134,"threads_started":1}
I20260619 02:43:34.935201 24966 raft_consensus.cc:2454] T 29ea8f6fab274720895167859ba05f78 P 8c50f55f75e24c12b576e81707dc182f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94dc4142e39d4affa12e736cb889ab46 in term 0.
I20260619 02:43:34.935434 25030 leader_election.cc:302] T 29ea8f6fab274720895167859ba05f78 P 94dc4142e39d4affa12e736cb889ab46 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c50f55f75e24c12b576e81707dc182f, 94dc4142e39d4affa12e736cb889ab46; no voters:
I20260619 02:43:34.935585 25165 raft_consensus.cc:2781] T 29ea8f6fab274720895167859ba05f78 P 94dc4142e39d4affa12e736cb889ab46 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:43:34.935581 24835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29ea8f6fab274720895167859ba05f78" candidate_uuid: "94dc4142e39d4affa12e736cb889ab46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acacf2b41c14469c8a450b400cd65333" is_pre_election: true
I20260619 02:43:34.935657 25165 raft_consensus.cc:482] T 29ea8f6fab274720895167859ba05f78 P 94dc4142e39d4affa12e736cb889ab46 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:43:34.935683 25165 raft_consensus.cc:3037] T 29ea8f6fab274720895167859ba05f78 P 94dc4142e39d4affa12e736cb889ab46 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:34.935724 24835 raft_consensus.cc:2454] T 29ea8f6fab274720895167859ba05f78 P acacf2b41c14469c8a450b400cd65333 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94dc4142e39d4affa12e736cb889ab46 in term 0.
I20260619 02:43:34.936467 25165 raft_consensus.cc:504] T 29ea8f6fab274720895167859ba05f78 P 94dc4142e39d4affa12e736cb889ab46 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acacf2b41c14469c8a450b400cd65333" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 36045 } } peers { permanent_uuid: "8c50f55f75e24c12b576e81707dc182f" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 40687 } } peers { permanent_uuid: "94dc4142e39d4affa12e736cb889ab46" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 33109 } }
I20260619 02:43:34.936614 25165 leader_election.cc:288] T 29ea8f6fab274720895167859ba05f78 P 94dc4142e39d4affa12e736cb889ab46 [CANDIDATE]: Term 1 election: Requested vote from peers acacf2b41c14469c8a450b400cd65333 (127.5.241.65:36045), 8c50f55f75e24c12b576e81707dc182f (127.5.241.66:40687)
I20260619 02:43:34.936758 24966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29ea8f6fab274720895167859ba05f78" candidate_uuid: "94dc4142e39d4affa12e736cb889ab46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c50f55f75e24c12b576e81707dc182f"
I20260619 02:43:34.936769 24835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29ea8f6fab274720895167859ba05f78" candidate_uuid: "94dc4142e39d4affa12e736cb889ab46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acacf2b41c14469c8a450b400cd65333"
I20260619 02:43:34.936838 24966 raft_consensus.cc:3037] T 29ea8f6fab274720895167859ba05f78 P 8c50f55f75e24c12b576e81707dc182f [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:34.936865 24835 raft_consensus.cc:3037] T 29ea8f6fab274720895167859ba05f78 P acacf2b41c14469c8a450b400cd65333 [term 0 FOLLOWER]: Advancing to term 1
W20260619 02:43:34.937011 25013 tablet.cc:2369] T 29ea8f6fab274720895167859ba05f78 P 8c50f55f75e24c12b576e81707dc182f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:43:34.937633 24966 raft_consensus.cc:2454] T 29ea8f6fab274720895167859ba05f78 P 8c50f55f75e24c12b576e81707dc182f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94dc4142e39d4affa12e736cb889ab46 in term 1.
I20260619 02:43:34.937742 24835 raft_consensus.cc:2454] T 29ea8f6fab274720895167859ba05f78 P acacf2b41c14469c8a450b400cd65333 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94dc4142e39d4affa12e736cb889ab46 in term 1.
I20260619 02:43:34.937813 25030 leader_election.cc:302] T 29ea8f6fab274720895167859ba05f78 P 94dc4142e39d4affa12e736cb889ab46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c50f55f75e24c12b576e81707dc182f, 94dc4142e39d4affa12e736cb889ab46; no voters:
I20260619 02:43:34.937938 25165 raft_consensus.cc:2781] T 29ea8f6fab274720895167859ba05f78 P 94dc4142e39d4affa12e736cb889ab46 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:43:34.938086 25165 raft_consensus.cc:686] T 29ea8f6fab274720895167859ba05f78 P 94dc4142e39d4affa12e736cb889ab46 [term 1 LEADER]: Becoming Leader. State: Replica: 94dc4142e39d4affa12e736cb889ab46, State: Running, Role: LEADER
I20260619 02:43:34.938184 25165 consensus_queue.cc:237] T 29ea8f6fab274720895167859ba05f78 P 94dc4142e39d4affa12e736cb889ab46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acacf2b41c14469c8a450b400cd65333" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 36045 } } peers { permanent_uuid: "8c50f55f75e24c12b576e81707dc182f" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 40687 } } peers { permanent_uuid: "94dc4142e39d4affa12e736cb889ab46" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 33109 } }
I20260619 02:43:34.939042 24692 catalog_manager.cc:5495] T 29ea8f6fab274720895167859ba05f78 P 94dc4142e39d4affa12e736cb889ab46 reported cstate change: term changed from 0 to 1, leader changed from <none> to 94dc4142e39d4affa12e736cb889ab46 (127.5.241.67). New cstate: current_term: 1 leader_uuid: "94dc4142e39d4affa12e736cb889ab46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acacf2b41c14469c8a450b400cd65333" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 36045 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c50f55f75e24c12b576e81707dc182f" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 40687 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94dc4142e39d4affa12e736cb889ab46" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 33109 } health_report { overall_health: HEALTHY } } }
I20260619 02:43:34.952327 25156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:34.949854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55244 (local address 127.5.241.67:33109)
0619 02:43:34.949919 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:34.949922 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:34.949932 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:43:34.949993 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:34.949994 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:34.950014 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:34.950085 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:43:34.950235 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.950705 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.951502 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:34.951713 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:34.951793 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:34.951952 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:43:34.952052 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:43:34.952167 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:34.952201 (+ 34us) server_negotiation.cc:299] Negotiation successful
0619 02:43:34.952209 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 02:43:34.954123 24835 raft_consensus.cc:1261] T 29ea8f6fab274720895167859ba05f78 P acacf2b41c14469c8a450b400cd65333 [term 1 FOLLOWER]: Refusing update from remote peer 94dc4142e39d4affa12e736cb889ab46: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:43:34.954159 24966 raft_consensus.cc:1261] T 29ea8f6fab274720895167859ba05f78 P 8c50f55f75e24c12b576e81707dc182f [term 1 FOLLOWER]: Refusing update from remote peer 94dc4142e39d4affa12e736cb889ab46: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:43:34.954546 25165 consensus_queue.cc:1035] T 29ea8f6fab274720895167859ba05f78 P 94dc4142e39d4affa12e736cb889ab46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acacf2b41c14469c8a450b400cd65333" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 36045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:43:34.954690 25167 consensus_queue.cc:1035] T 29ea8f6fab274720895167859ba05f78 P 94dc4142e39d4affa12e736cb889ab46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c50f55f75e24c12b576e81707dc182f" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 40687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:43:34.959568 24691 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37968:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:43:34.959650 24691 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37968:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:43:34.960381 24691 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cb28d9a8f0824c7e9e06d0e7fdd8b5cb: Sending DeleteTablet for 3 replicas of tablet 29ea8f6fab274720895167859ba05f78
I20260619 02:43:34.960700 24946 tablet_service.cc:1514] Processing DeleteTablet for tablet 29ea8f6fab274720895167859ba05f78 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:43:34 UTC) from {username='slave'} at 127.0.0.1:42746
I20260619 02:43:34.960753 25077 tablet_service.cc:1514] Processing DeleteTablet for tablet 29ea8f6fab274720895167859ba05f78 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:43:34 UTC) from {username='slave'} at 127.0.0.1:55228
I20260619 02:43:34.961047 24815 tablet_service.cc:1514] Processing DeleteTablet for tablet 29ea8f6fab274720895167859ba05f78 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:43:34 UTC) from {username='slave'} at 127.0.0.1:54420
I20260619 02:43:34.961283 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 24753
I20260619 02:43:34.966902 25181 tablet_replica.cc:330] T 29ea8f6fab274720895167859ba05f78 P 94dc4142e39d4affa12e736cb889ab46: stopping tablet replica
I20260619 02:43:34.967037 25181 raft_consensus.cc:2229] T 29ea8f6fab274720895167859ba05f78 P 94dc4142e39d4affa12e736cb889ab46 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:43:34.967156 25181 raft_consensus.cc:2258] T 29ea8f6fab274720895167859ba05f78 P 94dc4142e39d4affa12e736cb889ab46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:43:34.967187 25180 tablet_replica.cc:330] T 29ea8f6fab274720895167859ba05f78 P 8c50f55f75e24c12b576e81707dc182f: stopping tablet replica
I20260619 02:43:34.967561 25181 ts_tablet_manager.cc:1905] T 29ea8f6fab274720895167859ba05f78 P 94dc4142e39d4affa12e736cb889ab46: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:43:34.967566 25180 raft_consensus.cc:2229] T 29ea8f6fab274720895167859ba05f78 P 8c50f55f75e24c12b576e81707dc182f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:43:34.967746 25180 raft_consensus.cc:2258] T 29ea8f6fab274720895167859ba05f78 P 8c50f55f75e24c12b576e81707dc182f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:43:34.968225 25180 ts_tablet_manager.cc:1905] T 29ea8f6fab274720895167859ba05f78 P 8c50f55f75e24c12b576e81707dc182f: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 02:43:34.968329 24679 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.241.65:36045 (error 108)
I20260619 02:43:34.969038 25152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:34.968916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:36045 (local address 127.0.0.1:54426)
0619 02:43:34.968966 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:34.969005 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.65:36045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260619 02:43:34.969151 24679 catalog_manager.cc:4584] TS acacf2b41c14469c8a450b400cd65333 (127.5.241.65:36045): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 29ea8f6fab274720895167859ba05f78: Network error: Client connection negotiation failed: client connection to 127.5.241.65:36045: connect: Connection refused (error 111)
I20260619 02:43:34.969337 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 24884
I20260619 02:43:34.969870 25181 ts_tablet_manager.cc:1918] T 29ea8f6fab274720895167859ba05f78 P 94dc4142e39d4affa12e736cb889ab46: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:43:34.969950 25181 log.cc:1196] T 29ea8f6fab274720895167859ba05f78 P 94dc4142e39d4affa12e736cb889ab46: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836772852275-6085-0/minicluster-data/ts-2/wal/wals/29ea8f6fab274720895167859ba05f78
I20260619 02:43:34.970196 25181 ts_tablet_manager.cc:1939] T 29ea8f6fab274720895167859ba05f78 P 94dc4142e39d4affa12e736cb889ab46: Deleting consensus metadata
I20260619 02:43:34.970839 24676 catalog_manager.cc:4841] TS 94dc4142e39d4affa12e736cb889ab46 (127.5.241.67:33109): tablet 29ea8f6fab274720895167859ba05f78 (table test-table [id=9381fe7ebabe4827956ed0a1a270fb14]) successfully deleted
I20260619 02:43:34.974668 25152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:34.974533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.66:40687 (local address 127.0.0.1:42748)
0619 02:43:34.974600 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:34.974634 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.66:40687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260619 02:43:34.974730 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 25015
I20260619 02:43:34.979794 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 24661
2026-06-19T02:43:34Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (753 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:43:35 dist-test-slave-zpbg krb5kdc[25186](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:43:35 dist-test-slave-zpbg krb5kdc[25186](info): set up 2 sockets
Jun 19 02:43:35 dist-test-slave-zpbg krb5kdc[25186](info): commencing operation
krb5kdc: starting...
W20260619 02:43:36.018818 6085 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Jun 19 02:43:36 dist-test-slave-zpbg krb5kdc[25186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837016, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:43:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:43:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.241.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
I20260619 02:43:36.141831 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:34591
--webserver_interface=127.5.241.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:40823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:34591
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:43:36.221792 25199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:36.222059 25199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:36.222082 25199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:36.223223 25199 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:43:36.223295 25199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:36.223323 25199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:36.223346 25199 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:43:36.223361 25199 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:43:36.224905 25199 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:40823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:34591
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:34591
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.241.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25199
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:43:36.225091 25199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:36.225261 25199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:36.230386 25204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:36.230391 25205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:36.230400 25207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:36.230530 25199 server_base.cc:1034] running on GCE node
I20260619 02:43:36.230870 25199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:36.231207 25199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:36.232372 25199 hybrid_clock.cc:648] HybridClock initialized: now 1781837016232357 us; error 46 us; skew 500 ppm
Jun 19 02:43:36 dist-test-slave-zpbg krb5kdc[25186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837016, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:36.234846 25199 init.cc:377] Logged in from keytab as kudu/127.5.241.126@KRBTEST.COM (short username kudu)
I20260619 02:43:36.235339 25199 webserver.cc:460] Webserver started at http://127.5.241.126:38211/ using document root <none> and password file <none>
I20260619 02:43:36.235600 25199 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:36.235690 25199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:36.235879 25199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:43:36.236981 25199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/master-0/data/instance:
uuid: "cbb248136ee643248fa526b0d36663fa"
format_stamp: "Formatted at 2026-06-19 02:43:36 on dist-test-slave-zpbg"
server_key: "46114822b4a6873aefd511b0f59aa24f"
server_key_iv: "7731913bbe74f545d106e00e85b2e478"
server_key_version: "encryptionkey@0"
I20260619 02:43:36.237371 25199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/master-0/wal/instance:
uuid: "cbb248136ee643248fa526b0d36663fa"
format_stamp: "Formatted at 2026-06-19 02:43:36 on dist-test-slave-zpbg"
server_key: "46114822b4a6873aefd511b0f59aa24f"
server_key_iv: "7731913bbe74f545d106e00e85b2e478"
server_key_version: "encryptionkey@0"
I20260619 02:43:36.238734 25199 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 02:43:36.239569 25214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:36.239763 25199 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:43:36.239884 25199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "cbb248136ee643248fa526b0d36663fa"
format_stamp: "Formatted at 2026-06-19 02:43:36 on dist-test-slave-zpbg"
server_key: "46114822b4a6873aefd511b0f59aa24f"
server_key_iv: "7731913bbe74f545d106e00e85b2e478"
server_key_version: "encryptionkey@0"
I20260619 02:43:36.239971 25199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:36.254072 25199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:36.256124 25199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:36.256305 25199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:36.260581 25199 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.126:34591
I20260619 02:43:36.260614 25266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.126:34591 every 8 connection(s)
I20260619 02:43:36.260964 25199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:43:36.261809 25267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:36.264173 25267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbb248136ee643248fa526b0d36663fa: Bootstrap starting.
I20260619 02:43:36.264767 25267 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cbb248136ee643248fa526b0d36663fa: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:36.265062 25267 log.cc:824] T 00000000000000000000000000000000 P cbb248136ee643248fa526b0d36663fa: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:36.265794 25267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbb248136ee643248fa526b0d36663fa: No bootstrap required, opened a new log
I20260619 02:43:36.266644 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 25199
I20260619 02:43:36.266781 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/master-0/wal/instance
I20260619 02:43:36.266963 6085 external_mini_cluster.cc:1428] Setting key 6c3b62089e8cad10c5ff3b9adfb08865
I20260619 02:43:36.268191 25267 raft_consensus.cc:348] T 00000000000000000000000000000000 P cbb248136ee643248fa526b0d36663fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbb248136ee643248fa526b0d36663fa" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 34591 } }
I20260619 02:43:36.268373 25267 raft_consensus.cc:374] T 00000000000000000000000000000000 P cbb248136ee643248fa526b0d36663fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:36.268420 25267 raft_consensus.cc:729] T 00000000000000000000000000000000 P cbb248136ee643248fa526b0d36663fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbb248136ee643248fa526b0d36663fa, State: Initialized, Role: FOLLOWER
I20260619 02:43:36.268534 25267 consensus_queue.cc:260] T 00000000000000000000000000000000 P cbb248136ee643248fa526b0d36663fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbb248136ee643248fa526b0d36663fa" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 34591 } }
I20260619 02:43:36.268621 25267 raft_consensus.cc:388] T 00000000000000000000000000000000 P cbb248136ee643248fa526b0d36663fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:43:36.268666 25267 raft_consensus.cc:482] T 00000000000000000000000000000000 P cbb248136ee643248fa526b0d36663fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:43:36.268716 25267 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cbb248136ee643248fa526b0d36663fa [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:36.269794 25267 raft_consensus.cc:504] T 00000000000000000000000000000000 P cbb248136ee643248fa526b0d36663fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbb248136ee643248fa526b0d36663fa" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 34591 } }
I20260619 02:43:36.269881 25267 leader_election.cc:302] T 00000000000000000000000000000000 P cbb248136ee643248fa526b0d36663fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cbb248136ee643248fa526b0d36663fa; no voters:
I20260619 02:43:36.270044 25267 leader_election.cc:288] T 00000000000000000000000000000000 P cbb248136ee643248fa526b0d36663fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:43:36.270124 25272 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cbb248136ee643248fa526b0d36663fa [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:43:36.270300 25272 raft_consensus.cc:686] T 00000000000000000000000000000000 P cbb248136ee643248fa526b0d36663fa [term 1 LEADER]: Becoming Leader. State: Replica: cbb248136ee643248fa526b0d36663fa, State: Running, Role: LEADER
I20260619 02:43:36.270313 25267 sys_catalog.cc:564] T 00000000000000000000000000000000 P cbb248136ee643248fa526b0d36663fa [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:43:36.270399 25272 consensus_queue.cc:237] T 00000000000000000000000000000000 P cbb248136ee643248fa526b0d36663fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbb248136ee643248fa526b0d36663fa" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 34591 } }
I20260619 02:43:36.270742 25272 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbb248136ee643248fa526b0d36663fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader cbb248136ee643248fa526b0d36663fa. Latest consensus state: current_term: 1 leader_uuid: "cbb248136ee643248fa526b0d36663fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbb248136ee643248fa526b0d36663fa" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 34591 } } }
I20260619 02:43:36.270759 25273 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbb248136ee643248fa526b0d36663fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cbb248136ee643248fa526b0d36663fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbb248136ee643248fa526b0d36663fa" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 34591 } } }
I20260619 02:43:36.270879 25272 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbb248136ee643248fa526b0d36663fa [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:36.270884 25273 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbb248136ee643248fa526b0d36663fa [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:36.271068 25280 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:43:36.271512 25280 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 19 02:43:36 dist-test-slave-zpbg krb5kdc[25186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837016, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:43:36.276098 25280 catalog_manager.cc:1347] Generated new cluster ID: 0af53c0dcff24a4d8a5e428d1fa5c631
I20260619 02:43:36.276145 25280 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:43:36.278746 25271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:36.267491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57300 (local address 127.5.241.126:34591)
0619 02:43:36.267753 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:36.267763 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:36.267802 (+ 39us) server_negotiation.cc:407] Connection header received
0619 02:43:36.268146 (+ 344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:36.268156 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:36.268273 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:36.268430 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:36.269358 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:36.270526 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:36.271802 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:36.272408 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:36.272513 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:36.275113 (+ 2600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:36.275119 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:36.275120 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:36.275136 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:36.277251 (+ 2115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:36.277602 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:36.277605 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:36.277606 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:36.277726 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:36.277939 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:36.277940 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:36.277941 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:36.278178 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:36.278361 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:36.278431 (+ 70us) server_negotiation.cc:299] Negotiation successful
0619 02:43:36.278569 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":77,"threads_started":1}
I20260619 02:43:36.291433 25280 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:43:36.291985 25280 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:43:36.298148 25280 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cbb248136ee643248fa526b0d36663fa: Generated new TSK 0
I20260619 02:43:36.298343 25280 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.241.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab.
Entry for principal kudu/127.5.241.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab.
I20260619 02:43:36.330139 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.65:0
--local_ip_for_outbound_sockets=127.5.241.65
--webserver_interface=127.5.241.65
--webserver_port=0
--tserver_master_addrs=127.5.241.126:34591
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab
--principal=kudu/127.5.241.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:40823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:43:36.429724 25293 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:36.429913 25293 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:36.429937 25293 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:36.431162 25293 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:36.431201 25293 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:36.431228 25293 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.65
I20260619 02:43:36.432655 25293 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:40823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.65.keytab
--principal=kudu/127.5.241.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.241.65
--webserver_port=0
--tserver_master_addrs=127.5.241.126:34591
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25293
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.65
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:43:36.432895 25293 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:36.433139 25293 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:36.438149 25299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:36.438284 25301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:36.438360 25298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:36.438481 25293 server_base.cc:1034] running on GCE node
I20260619 02:43:36.438696 25293 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:36.438952 25293 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:36.440152 25293 hybrid_clock.cc:648] HybridClock initialized: now 1781837016440145 us; error 38 us; skew 500 ppm
Jun 19 02:43:36 dist-test-slave-zpbg krb5kdc[25186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837016, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:36.442679 25293 init.cc:377] Logged in from keytab as kudu/127.5.241.65@KRBTEST.COM (short username kudu)
I20260619 02:43:36.443266 25293 webserver.cc:460] Webserver started at http://127.5.241.65:46819/ using document root <none> and password file <none>
I20260619 02:43:36.443461 25293 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:36.443501 25293 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:36.443588 25293 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:43:36.444622 25293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/ts-0/data/instance:
uuid: "25c18816dfa54ddfbbfb39b17b3a6823"
format_stamp: "Formatted at 2026-06-19 02:43:36 on dist-test-slave-zpbg"
server_key: "852b01aa249a108959c6616cd5dd1a47"
server_key_iv: "ee2b1ca2bf1571f84c76b3ab1a22c633"
server_key_version: "encryptionkey@0"
I20260619 02:43:36.444903 25293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/ts-0/wal/instance:
uuid: "25c18816dfa54ddfbbfb39b17b3a6823"
format_stamp: "Formatted at 2026-06-19 02:43:36 on dist-test-slave-zpbg"
server_key: "852b01aa249a108959c6616cd5dd1a47"
server_key_iv: "ee2b1ca2bf1571f84c76b3ab1a22c633"
server_key_version: "encryptionkey@0"
I20260619 02:43:36.446175 25293 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:43:36.447054 25308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:36.447290 25293 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 02:43:36.447388 25293 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/ts-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/ts-0/wal
uuid: "25c18816dfa54ddfbbfb39b17b3a6823"
format_stamp: "Formatted at 2026-06-19 02:43:36 on dist-test-slave-zpbg"
server_key: "852b01aa249a108959c6616cd5dd1a47"
server_key_iv: "ee2b1ca2bf1571f84c76b3ab1a22c633"
server_key_version: "encryptionkey@0"
I20260619 02:43:36.447495 25293 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:36.468818 25293 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:36.471390 25293 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:36.471514 25293 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:36.471709 25293 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:43:36.472057 25293 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:43:36.472124 25293 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:36.472172 25293 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:43:36.472203 25293 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:36.478520 25293 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.65:33683
I20260619 02:43:36.478595 25421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.65:33683 every 8 connection(s)
I20260619 02:43:36.478988 25293 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
Jun 19 02:43:36 dist-test-slave-zpbg krb5kdc[25186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837016, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.65@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:43:36.486070 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 25293
I20260619 02:43:36.486198 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/ts-0/wal/instance
I20260619 02:43:36.486405 6085 external_mini_cluster.cc:1428] Setting key af012b800eb03aa373ec4b46fff7306d
I20260619 02:43:36.489315 25271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:36.479742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:42123 (local address 127.5.241.126:34591)
0619 02:43:36.479807 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:36.479809 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:36.480165 (+ 356us) server_negotiation.cc:407] Connection header received
0619 02:43:36.480634 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:36.480635 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:36.480648 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:36.480712 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:36.481691 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:36.482182 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:36.483321 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:36.483576 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:36.483649 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:36.485888 (+ 2239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:36.485889 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:36.485889 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:36.485898 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:36.487720 (+ 1822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:36.488202 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:36.488203 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:36.488203 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:36.488282 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:36.488643 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:36.488644 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:36.488644 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:36.488772 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:36.488849 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:36.489124 (+ 275us) server_negotiation.cc:299] Negotiation successful
0619 02:43:36.489217 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 02:43:36.489512 25424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:36.479700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:34591 (local address 127.5.241.65:42123)
0619 02:43:36.479898 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:36.479942 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:36.480522 (+ 580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:36.480790 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:36.480793 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:36.481400 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:43:36.481607 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:36.481612 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:36.482276 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:36.482276 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:36.483209 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:36.483212 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:36.483669 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:36.483670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:36.483782 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:36.484266 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:43:36.484275 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:43:36.485770 (+ 1495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:43:36.487857 (+ 2087us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:36.487864 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:36.487866 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:36.488120 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:36.488416 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:36.488417 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:36.488417 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:36.488529 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:36.488852 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:43:36.488860 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:43:36.488988 (+ 128us) client_negotiation.cc:769] Sending connection context
0619 02:43:36.489103 (+ 115us) client_negotiation.cc:240] Negotiation successful
0619 02:43:36.489266 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":123,"threads_started":1}
I20260619 02:43:36.490056 25422 heartbeater.cc:347] Connected to a master server at 127.5.241.126:34591
I20260619 02:43:36.490142 25422 heartbeater.cc:464] Registering TS with master...
I20260619 02:43:36.490350 25422 heartbeater.cc:511] Master 127.5.241.126:34591 requested a full tablet report, sending...
I20260619 02:43:36.490953 25231 ts_manager.cc:194] Registered new tserver with Master: 25c18816dfa54ddfbbfb39b17b3a6823 (127.5.241.65:33683)
I20260619 02:43:36.491567 25231 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.65@KRBTEST.COM'} at 127.5.241.65:42123
WARNING: no policy specified for kudu/127.5.241.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab.
Entry for principal kudu/127.5.241.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab.
I20260619 02:43:36.515055 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.66:0
--local_ip_for_outbound_sockets=127.5.241.66
--webserver_interface=127.5.241.66
--webserver_port=0
--tserver_master_addrs=127.5.241.126:34591
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab
--principal=kudu/127.5.241.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:40823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:43:36.594813 25427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:36.595053 25427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:36.595089 25427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:36.596225 25427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:36.596297 25427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:36.596341 25427 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.66
I20260619 02:43:36.597802 25427 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:40823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.66.keytab
--principal=kudu/127.5.241.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.241.66
--webserver_port=0
--tserver_master_addrs=127.5.241.126:34591
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25427
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.66
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:43:36.598062 25427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:36.598290 25427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:36.603562 25433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:36.603624 25435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:36.603624 25432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:36.603803 25427 server_base.cc:1034] running on GCE node
I20260619 02:43:36.604027 25427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:36.604272 25427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:36.605428 25427 hybrid_clock.cc:648] HybridClock initialized: now 1781837016605428 us; error 39 us; skew 500 ppm
Jun 19 02:43:36 dist-test-slave-zpbg krb5kdc[25186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837016, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:36.607852 25427 init.cc:377] Logged in from keytab as kudu/127.5.241.66@KRBTEST.COM (short username kudu)
I20260619 02:43:36.608294 25427 webserver.cc:460] Webserver started at http://127.5.241.66:34425/ using document root <none> and password file <none>
I20260619 02:43:36.608539 25427 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:36.608608 25427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:36.608757 25427 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:43:36.609828 25427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/ts-1/data/instance:
uuid: "7f2192f0b0e84bfeb6172e011c7aebf2"
format_stamp: "Formatted at 2026-06-19 02:43:36 on dist-test-slave-zpbg"
server_key: "d11e455ce35b52a3bd5085125bb347f7"
server_key_iv: "9fc2ccb865d6d6673f8767c551f79b7c"
server_key_version: "encryptionkey@0"
I20260619 02:43:36.610177 25427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/ts-1/wal/instance:
uuid: "7f2192f0b0e84bfeb6172e011c7aebf2"
format_stamp: "Formatted at 2026-06-19 02:43:36 on dist-test-slave-zpbg"
server_key: "d11e455ce35b52a3bd5085125bb347f7"
server_key_iv: "9fc2ccb865d6d6673f8767c551f79b7c"
server_key_version: "encryptionkey@0"
I20260619 02:43:36.611543 25427 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260619 02:43:36.612231 25442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:36.612406 25427 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:43:36.612504 25427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/ts-1/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/ts-1/wal
uuid: "7f2192f0b0e84bfeb6172e011c7aebf2"
format_stamp: "Formatted at 2026-06-19 02:43:36 on dist-test-slave-zpbg"
server_key: "d11e455ce35b52a3bd5085125bb347f7"
server_key_iv: "9fc2ccb865d6d6673f8767c551f79b7c"
server_key_version: "encryptionkey@0"
I20260619 02:43:36.612581 25427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:36.650684 25427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:36.652874 25427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:36.653045 25427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:36.653299 25427 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:43:36.653609 25427 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:43:36.653666 25427 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:36.653708 25427 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:43:36.653749 25427 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:36.659667 25427 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.66:39189
I20260619 02:43:36.659754 25555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.66:39189 every 8 connection(s)
I20260619 02:43:36.660077 25427 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
Jun 19 02:43:36 dist-test-slave-zpbg krb5kdc[25186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837016, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.66@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:43:36.669443 25271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:36.660952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:40603 (local address 127.5.241.126:34591)
0619 02:43:36.661046 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:36.661049 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:36.661186 (+ 137us) server_negotiation.cc:407] Connection header received
0619 02:43:36.661691 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:36.661692 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:36.661702 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:36.661762 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:36.662825 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:36.663390 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:36.664390 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:36.664572 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:36.664624 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:36.666713 (+ 2089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:36.666715 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:36.666715 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:36.666723 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:36.668210 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:36.668580 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:36.668580 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:36.668581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:36.668629 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:36.668843 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:36.668843 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:36.668844 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:36.668949 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:36.669003 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:36.669307 (+ 304us) server_negotiation.cc:299] Negotiation successful
0619 02:43:36.669375 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260619 02:43:36.669756 25558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:36.660864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:34591 (local address 127.5.241.66:40603)
0619 02:43:36.661040 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:36.661082 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:36.661588 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:36.661830 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:36.661833 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:36.662427 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:43:36.662646 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:36.662652 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:36.663490 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:36.663491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:36.664312 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:36.664314 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:36.664645 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:36.664645 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:36.664752 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:36.665221 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:43:36.665229 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:43:36.666625 (+ 1396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:43:36.668287 (+ 1662us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:36.668293 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:36.668295 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:36.668513 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:36.668686 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:36.668687 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:36.668687 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:36.668787 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:36.669037 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:43:36.669043 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:43:36.669185 (+ 142us) client_negotiation.cc:769] Sending connection context
0619 02:43:36.669287 (+ 102us) client_negotiation.cc:240] Negotiation successful
0619 02:43:36.669418 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":99,"threads_started":1}
I20260619 02:43:36.670197 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 25427
I20260619 02:43:36.670312 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/ts-1/wal/instance
I20260619 02:43:36.670441 25556 heartbeater.cc:347] Connected to a master server at 127.5.241.126:34591
I20260619 02:43:36.670538 25556 heartbeater.cc:464] Registering TS with master...
I20260619 02:43:36.670549 6085 external_mini_cluster.cc:1428] Setting key fb346f76c9717889977aaf3871996ddd
I20260619 02:43:36.670743 25556 heartbeater.cc:511] Master 127.5.241.126:34591 requested a full tablet report, sending...
I20260619 02:43:36.671190 25230 ts_manager.cc:194] Registered new tserver with Master: 7f2192f0b0e84bfeb6172e011c7aebf2 (127.5.241.66:39189)
I20260619 02:43:36.671712 25230 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.66@KRBTEST.COM'} at 127.5.241.66:40603
WARNING: no policy specified for kudu/127.5.241.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab.
Entry for principal kudu/127.5.241.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab.
I20260619 02:43:36.698571 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.67:0
--local_ip_for_outbound_sockets=127.5.241.67
--webserver_interface=127.5.241.67
--webserver_port=0
--tserver_master_addrs=127.5.241.126:34591
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab
--principal=kudu/127.5.241.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:40823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:43:36.779896 25561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:36.780133 25561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:36.780174 25561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:36.781317 25561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:36.781387 25561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:36.781432 25561 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.67
I20260619 02:43:36.782980 25561 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:40823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.67.keytab
--principal=kudu/127.5.241.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.241.67
--webserver_port=0
--tserver_master_addrs=127.5.241.126:34591
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.67
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:43:36.783233 25561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:36.783454 25561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:36.788324 25566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:36.788311 25569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:36.788344 25567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:36.788895 25561 server_base.cc:1034] running on GCE node
I20260619 02:43:36.789105 25561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:36.789359 25561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:36.790527 25561 hybrid_clock.cc:648] HybridClock initialized: now 1781837016790505 us; error 33 us; skew 500 ppm
Jun 19 02:43:36 dist-test-slave-zpbg krb5kdc[25186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837016, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:36.793066 25561 init.cc:377] Logged in from keytab as kudu/127.5.241.67@KRBTEST.COM (short username kudu)
I20260619 02:43:36.793551 25561 webserver.cc:460] Webserver started at http://127.5.241.67:44011/ using document root <none> and password file <none>
I20260619 02:43:36.793787 25561 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:36.793855 25561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:36.793991 25561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:43:36.795137 25561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/ts-2/data/instance:
uuid: "38dfd005d4224f438e81565dbae14f0a"
format_stamp: "Formatted at 2026-06-19 02:43:36 on dist-test-slave-zpbg"
server_key: "7ace9b27a8466a7689b8986ce97e282c"
server_key_iv: "1c1d5d995cc8705779165dec1b5e5f03"
server_key_version: "encryptionkey@0"
I20260619 02:43:36.795499 25561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/ts-2/wal/instance:
uuid: "38dfd005d4224f438e81565dbae14f0a"
format_stamp: "Formatted at 2026-06-19 02:43:36 on dist-test-slave-zpbg"
server_key: "7ace9b27a8466a7689b8986ce97e282c"
server_key_iv: "1c1d5d995cc8705779165dec1b5e5f03"
server_key_version: "encryptionkey@0"
I20260619 02:43:36.796855 25561 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:43:36.797593 25576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:36.797791 25561 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:43:36.797889 25561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/ts-2/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/ts-2/wal
uuid: "38dfd005d4224f438e81565dbae14f0a"
format_stamp: "Formatted at 2026-06-19 02:43:36 on dist-test-slave-zpbg"
server_key: "7ace9b27a8466a7689b8986ce97e282c"
server_key_iv: "1c1d5d995cc8705779165dec1b5e5f03"
server_key_version: "encryptionkey@0"
I20260619 02:43:36.797971 25561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:36.807204 25561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:36.809065 25561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:36.809222 25561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:36.809451 25561 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:43:36.809762 25561 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:43:36.809818 25561 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:36.809859 25561 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:43:36.809885 25561 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:36.815955 25561 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.67:36111
I20260619 02:43:36.815992 25689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.67:36111 every 8 connection(s)
I20260619 02:43:36.816315 25561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
Jun 19 02:43:36 dist-test-slave-zpbg krb5kdc[25186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837016, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.67@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:43:36.823462 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 25561
I20260619 02:43:36.823571 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/ts-2/wal/instance
I20260619 02:43:36.823763 6085 external_mini_cluster.cc:1428] Setting key 50e4b10d826c405ca392b246c3540206
I20260619 02:43:36.826159 25271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:36.817042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:34019 (local address 127.5.241.126:34591)
0619 02:43:36.817113 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:36.817115 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:36.817325 (+ 210us) server_negotiation.cc:407] Connection header received
0619 02:43:36.817858 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:36.817860 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:36.817872 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:36.817923 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:36.818945 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:36.819453 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:36.820529 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:36.820740 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:36.820834 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:36.823148 (+ 2314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:36.823150 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:36.823151 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:36.823163 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:36.824737 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:36.825177 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:36.825178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:36.825179 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:36.825241 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:36.825505 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:36.825507 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:36.825507 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:36.825634 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:36.825711 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:36.825984 (+ 273us) server_negotiation.cc:299] Negotiation successful
0619 02:43:36.826062 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 02:43:36.826360 25692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:36.817032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:34591 (local address 127.5.241.67:34019)
0619 02:43:36.817207 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:36.817240 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:36.817737 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:36.817993 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:36.817996 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:36.818547 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:43:36.818736 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:36.818740 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:36.819568 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:36.819569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:36.820437 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:36.820439 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:36.820842 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:36.820843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:36.820962 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:36.821481 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:43:36.821490 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:43:36.823035 (+ 1545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:43:36.824840 (+ 1805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:36.824846 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:36.824849 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:36.825085 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:36.825309 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:36.825310 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:36.825310 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:36.825427 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:36.825703 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:43:36.825709 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:43:36.825849 (+ 140us) client_negotiation.cc:769] Sending connection context
0619 02:43:36.825962 (+ 113us) client_negotiation.cc:240] Negotiation successful
0619 02:43:36.826129 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":92,"threads_started":1}
I20260619 02:43:36.826820 25690 heartbeater.cc:347] Connected to a master server at 127.5.241.126:34591
I20260619 02:43:36.826910 25690 heartbeater.cc:464] Registering TS with master...
I20260619 02:43:36.827100 25690 heartbeater.cc:511] Master 127.5.241.126:34591 requested a full tablet report, sending...
I20260619 02:43:36.827459 25230 ts_manager.cc:194] Registered new tserver with Master: 38dfd005d4224f438e81565dbae14f0a (127.5.241.67:36111)
I20260619 02:43:36.827843 25230 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.241.67@KRBTEST.COM'} at 127.5.241.67:34019
I20260619 02:43:36.835597 6085 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:43:36.843365 25271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:36.836962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57310 (local address 127.5.241.126:34591)
0619 02:43:36.837054 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:36.837057 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:36.837241 (+ 184us) server_negotiation.cc:407] Connection header received
0619 02:43:36.837332 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:36.837333 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:36.837343 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:36.837418 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:36.837987 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:36.838475 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:36.839331 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:36.839521 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:36.839610 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:36.840820 (+ 1210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:36.840821 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:36.840822 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:36.840828 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:36.842010 (+ 1182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:36.842421 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:36.842422 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:36.842423 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:36.842482 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:36.842692 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:36.842693 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:36.842694 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:36.842918 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:36.843026 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:36.843189 (+ 163us) server_negotiation.cc:299] Negotiation successful
0619 02:43:36.843264 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 02:43:36.845067 25230 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57310:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:43:36.846693 25230 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:43:36.853621 25702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:36.849268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.66:39189 (local address 127.0.0.1:41456)
0619 02:43:36.849398 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:36.849407 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:36.849470 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:36.849795 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:36.849799 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:36.849802 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:43:36.849873 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:36.849875 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:36.851276 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:36.851278 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:36.852374 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:36.852377 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:36.853288 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:36.853289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:36.853377 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:36.853393 (+ 16us) client_negotiation.cc:769] Sending connection context
0619 02:43:36.853474 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 02:43:36.853530 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":38,"thread_start_us":81,"threads_started":1}
I20260619 02:43:36.853654 25703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:36.849146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43982 (local address 127.5.241.65:33683)
0619 02:43:36.849910 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:36.849914 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:36.849923 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:43:36.849990 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:36.849992 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:36.850008 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:36.850076 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:43:36.850224 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:36.851164 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:36.852623 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:36.853327 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:36.853390 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:36.853440 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:36.853530 (+ 90us) server_negotiation.cc:299] Negotiation successful
0619 02:43:36.853569 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":724,"thread_start_us":99,"threads_started":1}
I20260619 02:43:36.853803 25705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:36.849045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.67:36111 (local address 127.0.0.1:44114)
0619 02:43:36.849393 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:36.849405 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:36.849471 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:36.850400 (+ 929us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:36.850401 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:36.850403 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:43:36.850443 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:36.850444 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:36.851737 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:36.851738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:36.852761 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:36.852763 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:36.853568 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:36.853569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:36.853627 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:36.853631 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:43:36.853679 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 02:43:36.853724 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":81,"threads_started":1}
I20260619 02:43:36.853858 25701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:36.849043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:33683 (local address 127.0.0.1:43982)
0619 02:43:36.849355 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:36.849402 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:36.849470 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:36.850089 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:36.850090 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:36.850092 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:43:36.850141 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:36.850142 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:36.851257 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:36.851259 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:36.852534 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:36.852537 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:36.853392 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:36.853393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:36.853456 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:36.853460 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:43:36.853517 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 02:43:36.853563 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":66,"threads_started":1}
I20260619 02:43:36.854100 25704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:36.849161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44114 (local address 127.5.241.67:36111)
0619 02:43:36.850213 (+ 1052us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:36.850217 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:36.850226 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:43:36.850263 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:36.850265 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:36.850280 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:36.850361 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:43:36.850528 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:36.851613 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:36.852844 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:36.853423 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:36.853482 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:36.853535 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:36.853980 (+ 445us) server_negotiation.cc:299] Negotiation successful
0619 02:43:36.854025 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1014,"thread_start_us":115,"threads_started":1}
I20260619 02:43:36.854514 25706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:36.849391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41456 (local address 127.5.241.66:39189)
0619 02:43:36.849581 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:36.849585 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:36.849596 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:43:36.849662 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:36.849664 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:36.849677 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:36.849742 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:43:36.849962 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:36.851164 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:36.852468 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:36.853182 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:36.853261 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:36.853321 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:36.854391 (+ 1070us) server_negotiation.cc:299] Negotiation successful
0619 02:43:36.854436 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":75,"threads_started":1}
I20260619 02:43:36.854848 25356 tablet_service.cc:1467] Processing CreateTablet for tablet f54d8f8ee33a48b7aab6cd35e2a615e6 (DEFAULT_TABLE table=test-table [id=ffebae11315d42599ee918cf64cdaa8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:43:36.855111 25490 tablet_service.cc:1467] Processing CreateTablet for tablet f54d8f8ee33a48b7aab6cd35e2a615e6 (DEFAULT_TABLE table=test-table [id=ffebae11315d42599ee918cf64cdaa8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:43:36.855238 25356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f54d8f8ee33a48b7aab6cd35e2a615e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:36.855196 25624 tablet_service.cc:1467] Processing CreateTablet for tablet f54d8f8ee33a48b7aab6cd35e2a615e6 (DEFAULT_TABLE table=test-table [id=ffebae11315d42599ee918cf64cdaa8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:43:36.855448 25490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f54d8f8ee33a48b7aab6cd35e2a615e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:36.855506 25624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f54d8f8ee33a48b7aab6cd35e2a615e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:36.857864 25707 tablet_bootstrap.cc:492] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 7f2192f0b0e84bfeb6172e011c7aebf2: Bootstrap starting.
I20260619 02:43:36.858685 25707 tablet_bootstrap.cc:654] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 7f2192f0b0e84bfeb6172e011c7aebf2: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:36.858657 25709 tablet_bootstrap.cc:492] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 25c18816dfa54ddfbbfb39b17b3a6823: Bootstrap starting.
I20260619 02:43:36.859395 25709 tablet_bootstrap.cc:654] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 25c18816dfa54ddfbbfb39b17b3a6823: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:36.859422 25707 log.cc:824] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 7f2192f0b0e84bfeb6172e011c7aebf2: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:36.859949 25709 log.cc:824] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 25c18816dfa54ddfbbfb39b17b3a6823: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:36.860217 25707 tablet_bootstrap.cc:492] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 7f2192f0b0e84bfeb6172e011c7aebf2: No bootstrap required, opened a new log
I20260619 02:43:36.860291 25707 ts_tablet_manager.cc:1397] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 7f2192f0b0e84bfeb6172e011c7aebf2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:43:36.860289 25708 tablet_bootstrap.cc:492] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 38dfd005d4224f438e81565dbae14f0a: Bootstrap starting.
I20260619 02:43:36.860889 25708 tablet_bootstrap.cc:654] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 38dfd005d4224f438e81565dbae14f0a: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:36.861419 25708 log.cc:824] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 38dfd005d4224f438e81565dbae14f0a: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:36.861940 25709 tablet_bootstrap.cc:492] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 25c18816dfa54ddfbbfb39b17b3a6823: No bootstrap required, opened a new log
I20260619 02:43:36.862012 25709 ts_tablet_manager.cc:1397] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 25c18816dfa54ddfbbfb39b17b3a6823: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260619 02:43:36.862147 25708 tablet_bootstrap.cc:492] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 38dfd005d4224f438e81565dbae14f0a: No bootstrap required, opened a new log
I20260619 02:43:36.862211 25708 ts_tablet_manager.cc:1397] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 38dfd005d4224f438e81565dbae14f0a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:43:36.862520 25707 raft_consensus.cc:348] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 7f2192f0b0e84bfeb6172e011c7aebf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25c18816dfa54ddfbbfb39b17b3a6823" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 33683 } } peers { permanent_uuid: "38dfd005d4224f438e81565dbae14f0a" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36111 } } peers { permanent_uuid: "7f2192f0b0e84bfeb6172e011c7aebf2" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 39189 } }
I20260619 02:43:36.862704 25707 raft_consensus.cc:374] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 7f2192f0b0e84bfeb6172e011c7aebf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:36.862742 25707 raft_consensus.cc:729] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 7f2192f0b0e84bfeb6172e011c7aebf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f2192f0b0e84bfeb6172e011c7aebf2, State: Initialized, Role: FOLLOWER
I20260619 02:43:36.862917 25707 consensus_queue.cc:260] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 7f2192f0b0e84bfeb6172e011c7aebf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25c18816dfa54ddfbbfb39b17b3a6823" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 33683 } } peers { permanent_uuid: "38dfd005d4224f438e81565dbae14f0a" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36111 } } peers { permanent_uuid: "7f2192f0b0e84bfeb6172e011c7aebf2" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 39189 } }
I20260619 02:43:36.863191 25707 ts_tablet_manager.cc:1428] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 7f2192f0b0e84bfeb6172e011c7aebf2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 02:43:36.863313 25556 heartbeater.cc:503] Master 127.5.241.126:34591 was elected leader, sending a full tablet report...
I20260619 02:43:36.863986 25709 raft_consensus.cc:348] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 25c18816dfa54ddfbbfb39b17b3a6823 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25c18816dfa54ddfbbfb39b17b3a6823" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 33683 } } peers { permanent_uuid: "38dfd005d4224f438e81565dbae14f0a" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36111 } } peers { permanent_uuid: "7f2192f0b0e84bfeb6172e011c7aebf2" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 39189 } }
I20260619 02:43:36.864146 25709 raft_consensus.cc:374] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 25c18816dfa54ddfbbfb39b17b3a6823 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:36.864183 25709 raft_consensus.cc:729] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 25c18816dfa54ddfbbfb39b17b3a6823 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25c18816dfa54ddfbbfb39b17b3a6823, State: Initialized, Role: FOLLOWER
I20260619 02:43:36.864315 25709 consensus_queue.cc:260] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 25c18816dfa54ddfbbfb39b17b3a6823 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25c18816dfa54ddfbbfb39b17b3a6823" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 33683 } } peers { permanent_uuid: "38dfd005d4224f438e81565dbae14f0a" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36111 } } peers { permanent_uuid: "7f2192f0b0e84bfeb6172e011c7aebf2" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 39189 } }
I20260619 02:43:36.864234 25708 raft_consensus.cc:348] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 38dfd005d4224f438e81565dbae14f0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25c18816dfa54ddfbbfb39b17b3a6823" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 33683 } } peers { permanent_uuid: "38dfd005d4224f438e81565dbae14f0a" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36111 } } peers { permanent_uuid: "7f2192f0b0e84bfeb6172e011c7aebf2" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 39189 } }
I20260619 02:43:36.864423 25708 raft_consensus.cc:374] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 38dfd005d4224f438e81565dbae14f0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:36.864472 25708 raft_consensus.cc:729] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 38dfd005d4224f438e81565dbae14f0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38dfd005d4224f438e81565dbae14f0a, State: Initialized, Role: FOLLOWER
I20260619 02:43:36.864593 25708 consensus_queue.cc:260] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 38dfd005d4224f438e81565dbae14f0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25c18816dfa54ddfbbfb39b17b3a6823" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 33683 } } peers { permanent_uuid: "38dfd005d4224f438e81565dbae14f0a" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36111 } } peers { permanent_uuid: "7f2192f0b0e84bfeb6172e011c7aebf2" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 39189 } }
I20260619 02:43:36.864653 25709 ts_tablet_manager.cc:1428] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 25c18816dfa54ddfbbfb39b17b3a6823: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260619 02:43:36.864698 25422 heartbeater.cc:503] Master 127.5.241.126:34591 was elected leader, sending a full tablet report...
I20260619 02:43:36.864871 25708 ts_tablet_manager.cc:1428] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 38dfd005d4224f438e81565dbae14f0a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 02:43:36.864903 25690 heartbeater.cc:503] Master 127.5.241.126:34591 was elected leader, sending a full tablet report...
W20260619 02:43:36.910667 25557 tablet.cc:2369] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 7f2192f0b0e84bfeb6172e011c7aebf2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:43:36.945510 25713 raft_consensus.cc:482] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 7f2192f0b0e84bfeb6172e011c7aebf2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:43:36.945680 25713 raft_consensus.cc:504] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 7f2192f0b0e84bfeb6172e011c7aebf2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25c18816dfa54ddfbbfb39b17b3a6823" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 33683 } } peers { permanent_uuid: "38dfd005d4224f438e81565dbae14f0a" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36111 } } peers { permanent_uuid: "7f2192f0b0e84bfeb6172e011c7aebf2" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 39189 } }
I20260619 02:43:36.946116 25713 leader_election.cc:288] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 7f2192f0b0e84bfeb6172e011c7aebf2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25c18816dfa54ddfbbfb39b17b3a6823 (127.5.241.65:33683), 38dfd005d4224f438e81565dbae14f0a (127.5.241.67:36111)
I20260619 02:43:36.949223 25703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:36.946496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:42709 (local address 127.5.241.65:33683)
0619 02:43:36.946555 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:36.946557 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:36.946650 (+ 93us) server_negotiation.cc:407] Connection header received
0619 02:43:36.946713 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:36.946714 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:36.946722 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:36.946785 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:43:36.946964 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:36.947451 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:36.948472 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:36.948851 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:36.948943 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:36.948952 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:36.949111 (+ 159us) server_negotiation.cc:299] Negotiation successful
0619 02:43:36.949148 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 02:43:36.949229 25704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:36.946552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:56531 (local address 127.5.241.67:36111)
0619 02:43:36.946612 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:36.946614 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:36.946623 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:43:36.946654 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:36.946655 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:36.946664 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:36.946722 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:43:36.946987 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:36.947455 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:36.948478 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:36.948865 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:36.948963 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:36.948972 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:36.949119 (+ 147us) server_negotiation.cc:299] Negotiation successful
0619 02:43:36.949157 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 02:43:36.949237 25558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:36.946327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.67:36111 (local address 127.5.241.66:56531)
0619 02:43:36.946402 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:36.946415 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:36.946458 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:36.946831 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:36.946832 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:36.946833 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:43:36.946885 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:36.946886 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:36.947542 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:36.947543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:36.948380 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:36.948383 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:36.948963 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:36.948964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:36.949037 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:36.949040 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:43:36.949113 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 02:43:36.949160 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":58}
I20260619 02:43:36.949247 25716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:36.946327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:33683 (local address 127.5.241.66:42709)
0619 02:43:36.946552 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:36.946564 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:36.946646 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:36.946804 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:36.946805 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:36.946806 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:43:36.946885 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:36.946886 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:36.947539 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:36.947540 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:36.948373 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:36.948375 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:36.948943 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:36.948944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:36.949014 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:36.949021 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 02:43:36.949106 (+ 85us) client_negotiation.cc:240] Negotiation successful
0619 02:43:36.949148 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":56,"thread_start_us":114,"threads_started":1}
I20260619 02:43:36.949695 25644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f54d8f8ee33a48b7aab6cd35e2a615e6" candidate_uuid: "7f2192f0b0e84bfeb6172e011c7aebf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38dfd005d4224f438e81565dbae14f0a" is_pre_election: true
I20260619 02:43:36.949918 25644 raft_consensus.cc:2454] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 38dfd005d4224f438e81565dbae14f0a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f2192f0b0e84bfeb6172e011c7aebf2 in term 0.
I20260619 02:43:36.949894 25376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f54d8f8ee33a48b7aab6cd35e2a615e6" candidate_uuid: "7f2192f0b0e84bfeb6172e011c7aebf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25c18816dfa54ddfbbfb39b17b3a6823" is_pre_election: true
I20260619 02:43:36.950052 25376 raft_consensus.cc:2454] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 25c18816dfa54ddfbbfb39b17b3a6823 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f2192f0b0e84bfeb6172e011c7aebf2 in term 0.
I20260619 02:43:36.950198 25446 leader_election.cc:302] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 7f2192f0b0e84bfeb6172e011c7aebf2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38dfd005d4224f438e81565dbae14f0a, 7f2192f0b0e84bfeb6172e011c7aebf2; no voters:
I20260619 02:43:36.950333 25713 raft_consensus.cc:2781] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 7f2192f0b0e84bfeb6172e011c7aebf2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:43:36.950415 25713 raft_consensus.cc:482] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 7f2192f0b0e84bfeb6172e011c7aebf2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:43:36.950449 25713 raft_consensus.cc:3037] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 7f2192f0b0e84bfeb6172e011c7aebf2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:36.951195 25713 raft_consensus.cc:504] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 7f2192f0b0e84bfeb6172e011c7aebf2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25c18816dfa54ddfbbfb39b17b3a6823" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 33683 } } peers { permanent_uuid: "38dfd005d4224f438e81565dbae14f0a" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36111 } } peers { permanent_uuid: "7f2192f0b0e84bfeb6172e011c7aebf2" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 39189 } }
I20260619 02:43:36.951365 25713 leader_election.cc:288] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 7f2192f0b0e84bfeb6172e011c7aebf2 [CANDIDATE]: Term 1 election: Requested vote from peers 25c18816dfa54ddfbbfb39b17b3a6823 (127.5.241.65:33683), 38dfd005d4224f438e81565dbae14f0a (127.5.241.67:36111)
I20260619 02:43:36.951509 25376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f54d8f8ee33a48b7aab6cd35e2a615e6" candidate_uuid: "7f2192f0b0e84bfeb6172e011c7aebf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25c18816dfa54ddfbbfb39b17b3a6823"
I20260619 02:43:36.951601 25376 raft_consensus.cc:3037] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 25c18816dfa54ddfbbfb39b17b3a6823 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:36.951613 25644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f54d8f8ee33a48b7aab6cd35e2a615e6" candidate_uuid: "7f2192f0b0e84bfeb6172e011c7aebf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38dfd005d4224f438e81565dbae14f0a"
I20260619 02:43:36.951709 25644 raft_consensus.cc:3037] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 38dfd005d4224f438e81565dbae14f0a [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:36.952279 25376 raft_consensus.cc:2454] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 25c18816dfa54ddfbbfb39b17b3a6823 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f2192f0b0e84bfeb6172e011c7aebf2 in term 1.
I20260619 02:43:36.952450 25444 leader_election.cc:302] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 7f2192f0b0e84bfeb6172e011c7aebf2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25c18816dfa54ddfbbfb39b17b3a6823, 7f2192f0b0e84bfeb6172e011c7aebf2; no voters:
I20260619 02:43:36.952548 25644 raft_consensus.cc:2454] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 38dfd005d4224f438e81565dbae14f0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f2192f0b0e84bfeb6172e011c7aebf2 in term 1.
I20260619 02:43:36.952559 25713 raft_consensus.cc:2781] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 7f2192f0b0e84bfeb6172e011c7aebf2 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:43:36.952708 25713 raft_consensus.cc:686] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 7f2192f0b0e84bfeb6172e011c7aebf2 [term 1 LEADER]: Becoming Leader. State: Replica: 7f2192f0b0e84bfeb6172e011c7aebf2, State: Running, Role: LEADER
I20260619 02:43:36.952807 25713 consensus_queue.cc:237] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 7f2192f0b0e84bfeb6172e011c7aebf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25c18816dfa54ddfbbfb39b17b3a6823" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 33683 } } peers { permanent_uuid: "38dfd005d4224f438e81565dbae14f0a" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36111 } } peers { permanent_uuid: "7f2192f0b0e84bfeb6172e011c7aebf2" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 39189 } }
I20260619 02:43:36.953591 25230 catalog_manager.cc:5495] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 7f2192f0b0e84bfeb6172e011c7aebf2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7f2192f0b0e84bfeb6172e011c7aebf2 (127.5.241.66). New cstate: current_term: 1 leader_uuid: "7f2192f0b0e84bfeb6172e011c7aebf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25c18816dfa54ddfbbfb39b17b3a6823" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 33683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38dfd005d4224f438e81565dbae14f0a" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f2192f0b0e84bfeb6172e011c7aebf2" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 39189 } health_report { overall_health: HEALTHY } } }
I20260619 02:43:36.971465 25706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:36.968812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41458 (local address 127.5.241.66:39189)
0619 02:43:36.968860 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:36.968862 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:36.968874 (+ 12us) server_negotiation.cc:407] Connection header received
0619 02:43:36.968920 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:36.968921 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:36.968942 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:36.969038 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:43:36.969201 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:36.969685 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:36.970573 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:36.970804 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:36.970887 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:36.971067 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:43:36.971182 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:43:36.971282 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:36.971302 (+ 20us) server_negotiation.cc:299] Negotiation successful
0619 02:43:36.971342 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 02:43:36.973377 25644 raft_consensus.cc:1261] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 38dfd005d4224f438e81565dbae14f0a [term 1 FOLLOWER]: Refusing update from remote peer 7f2192f0b0e84bfeb6172e011c7aebf2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:43:36.973400 25376 raft_consensus.cc:1261] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 25c18816dfa54ddfbbfb39b17b3a6823 [term 1 FOLLOWER]: Refusing update from remote peer 7f2192f0b0e84bfeb6172e011c7aebf2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:43:36.973774 25717 consensus_queue.cc:1035] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 7f2192f0b0e84bfeb6172e011c7aebf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38dfd005d4224f438e81565dbae14f0a" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 36111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:43:36.973920 25713 consensus_queue.cc:1035] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 7f2192f0b0e84bfeb6172e011c7aebf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25c18816dfa54ddfbbfb39b17b3a6823" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:43:36.976944 25721 mvcc.cc:205] Tried to move back new op lower bound from 7298404421519949824 to 7298404421439213568. Current Snapshot: MvccSnapshot[applied={T|T < 7298404421519949824}]
I20260619 02:43:36.978833 25230 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57310:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:43:36.978942 25230 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57310:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:43:36.979321 25722 mvcc.cc:205] Tried to move back new op lower bound from 7298404421519949824 to 7298404421439213568. Current Snapshot: MvccSnapshot[applied={T|T < 7298404421519949824 or (T in {7298404421519949824})}]
I20260619 02:43:36.979760 25230 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cbb248136ee643248fa526b0d36663fa: Sending DeleteTablet for 3 replicas of tablet f54d8f8ee33a48b7aab6cd35e2a615e6
I20260619 02:43:36.980090 25624 tablet_service.cc:1514] Processing DeleteTablet for tablet f54d8f8ee33a48b7aab6cd35e2a615e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:43:36 UTC) from {username='kudu', principal='kudu/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:44114
I20260619 02:43:36.980175 25356 tablet_service.cc:1514] Processing DeleteTablet for tablet f54d8f8ee33a48b7aab6cd35e2a615e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:43:36 UTC) from {username='kudu', principal='kudu/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:43982
I20260619 02:43:36.980310 25729 tablet_replica.cc:330] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 38dfd005d4224f438e81565dbae14f0a: stopping tablet replica
I20260619 02:43:36.980419 25730 tablet_replica.cc:330] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 25c18816dfa54ddfbbfb39b17b3a6823: stopping tablet replica
I20260619 02:43:36.980398 25490 tablet_service.cc:1514] Processing DeleteTablet for tablet f54d8f8ee33a48b7aab6cd35e2a615e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:43:36 UTC) from {username='kudu', principal='kudu/127.5.241.126@KRBTEST.COM'} at 127.0.0.1:41456
I20260619 02:43:36.980432 25729 raft_consensus.cc:2229] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 38dfd005d4224f438e81565dbae14f0a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:43:36.980499 25730 raft_consensus.cc:2229] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 25c18816dfa54ddfbbfb39b17b3a6823 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:43:36.980533 25729 raft_consensus.cc:2258] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 38dfd005d4224f438e81565dbae14f0a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:43:36.980576 25730 raft_consensus.cc:2258] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 25c18816dfa54ddfbbfb39b17b3a6823 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:43:36.980729 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 25293
I20260619 02:43:36.981734 25729 ts_tablet_manager.cc:1905] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 38dfd005d4224f438e81565dbae14f0a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:43:36.983479 25729 ts_tablet_manager.cc:1918] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 38dfd005d4224f438e81565dbae14f0a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:43:36.983593 25729 log.cc:1196] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 38dfd005d4224f438e81565dbae14f0a: Deleting WAL directory at /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836772852275-6085-0/minicluster-data/ts-2/wal/wals/f54d8f8ee33a48b7aab6cd35e2a615e6
I20260619 02:43:36.983920 25729 ts_tablet_manager.cc:1939] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 38dfd005d4224f438e81565dbae14f0a: Deleting consensus metadata
I20260619 02:43:36.984344 25218 catalog_manager.cc:4841] TS 38dfd005d4224f438e81565dbae14f0a (127.5.241.67:36111): tablet f54d8f8ee33a48b7aab6cd35e2a615e6 (table test-table [id=ffebae11315d42599ee918cf64cdaa8a]) successfully deleted
W20260619 02:43:36.986831 25216 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.241.65:33683 (error 108)
I20260619 02:43:36.986900 25731 tablet_replica.cc:330] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 7f2192f0b0e84bfeb6172e011c7aebf2: stopping tablet replica
I20260619 02:43:36.987038 25731 raft_consensus.cc:2229] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 7f2192f0b0e84bfeb6172e011c7aebf2 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:43:36.987149 25731 raft_consensus.cc:2258] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 7f2192f0b0e84bfeb6172e011c7aebf2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:43:36.987296 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 25427
I20260619 02:43:36.987571 25731 ts_tablet_manager.cc:1905] T f54d8f8ee33a48b7aab6cd35e2a615e6 P 7f2192f0b0e84bfeb6172e011c7aebf2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:43:36.988909 25702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:36.988774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:33683 (local address 127.0.0.1:43992)
0619 02:43:36.988836 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:36.988879 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.65:33683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260619 02:43:36.990890 25216 catalog_manager.cc:4584] TS 25c18816dfa54ddfbbfb39b17b3a6823 (127.5.241.65:33683): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f54d8f8ee33a48b7aab6cd35e2a615e6: Network error: Client connection negotiation failed: client connection to 127.5.241.65:33683: connect: Connection refused (error 111)
I20260619 02:43:36.992614 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 25561
I20260619 02:43:36.992699 25702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:36.992562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.66:39189 (local address 127.0.0.1:41468)
0619 02:43:36.992620 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:36.992648 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.66:39189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 02:43:36.997133 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 25199
2026-06-19T02:43:37Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2019 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-06-19T02:43:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:43:37Z Disabled control of system clock
I20260619 02:43:37.026356 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:45389
--webserver_interface=127.5.241.126
--webserver_port=0
--builtin_ntp_servers=127.5.241.84:44391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:45389
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 02:43:37.124523 25736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:37.124752 25736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:37.124773 25736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:37.125936 25736 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:43:37.125978 25736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:37.125991 25736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:37.126004 25736 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:43:37.126016 25736 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:43:37.127467 25736 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:44391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:45389
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:45389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.241.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:43:37.127723 25736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:37.128010 25736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:37.132989 25742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:37.132956 25744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:37.133081 25736 server_base.cc:1034] running on GCE node
W20260619 02:43:37.132969 25741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:37.133424 25736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:37.133736 25736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:37.134933 25736 hybrid_clock.cc:648] HybridClock initialized: now 1781837017134934 us; error 45 us; skew 500 ppm
I20260619 02:43:37.136282 25736 webserver.cc:460] Webserver started at http://127.5.241.126:45939/ using document root <none> and password file <none>
I20260619 02:43:37.136576 25736 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:37.136657 25736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:37.136826 25736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:43:37.137970 25736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/master-0/data/instance:
uuid: "cc5c8fbb4cbc4f78919c978e4d2a7ce4"
format_stamp: "Formatted at 2026-06-19 02:43:37 on dist-test-slave-zpbg"
server_key: "39b7cdacb8a8eb54ff62d06f5f7dee4d"
server_key_iv: "ecf3540300c46e17c85dfd0818da50ae"
server_key_version: "encryptionkey@0"
I20260619 02:43:37.138352 25736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/master-0/wal/instance:
uuid: "cc5c8fbb4cbc4f78919c978e4d2a7ce4"
format_stamp: "Formatted at 2026-06-19 02:43:37 on dist-test-slave-zpbg"
server_key: "39b7cdacb8a8eb54ff62d06f5f7dee4d"
server_key_iv: "ecf3540300c46e17c85dfd0818da50ae"
server_key_version: "encryptionkey@0"
I20260619 02:43:37.139854 25736 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:43:37.140707 25750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:37.140910 25736 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:43:37.141013 25736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "cc5c8fbb4cbc4f78919c978e4d2a7ce4"
format_stamp: "Formatted at 2026-06-19 02:43:37 on dist-test-slave-zpbg"
server_key: "39b7cdacb8a8eb54ff62d06f5f7dee4d"
server_key_iv: "ecf3540300c46e17c85dfd0818da50ae"
server_key_version: "encryptionkey@0"
I20260619 02:43:37.141098 25736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:37.164181 25736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:37.164520 25736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:37.164675 25736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:37.168490 25736 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.126:45389
I20260619 02:43:37.168530 25802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.126:45389 every 8 connection(s)
I20260619 02:43:37.168898 25736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:43:37.169914 25803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:37.171002 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 25736
I20260619 02:43:37.171128 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/master-0/wal/instance
I20260619 02:43:37.171329 6085 external_mini_cluster.cc:1428] Setting key 139de7869282c17ed548fa457557c467
I20260619 02:43:37.172547 25803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc5c8fbb4cbc4f78919c978e4d2a7ce4: Bootstrap starting.
I20260619 02:43:37.173259 25803 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc5c8fbb4cbc4f78919c978e4d2a7ce4: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:37.173581 25803 log.cc:824] T 00000000000000000000000000000000 P cc5c8fbb4cbc4f78919c978e4d2a7ce4: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:37.174314 25803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc5c8fbb4cbc4f78919c978e4d2a7ce4: No bootstrap required, opened a new log
I20260619 02:43:37.176499 25803 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc5c8fbb4cbc4f78919c978e4d2a7ce4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc5c8fbb4cbc4f78919c978e4d2a7ce4" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45389 } }
I20260619 02:43:37.176680 25803 raft_consensus.cc:374] T 00000000000000000000000000000000 P cc5c8fbb4cbc4f78919c978e4d2a7ce4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:37.176719 25803 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc5c8fbb4cbc4f78919c978e4d2a7ce4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc5c8fbb4cbc4f78919c978e4d2a7ce4, State: Initialized, Role: FOLLOWER
I20260619 02:43:37.176848 25803 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc5c8fbb4cbc4f78919c978e4d2a7ce4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc5c8fbb4cbc4f78919c978e4d2a7ce4" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45389 } }
I20260619 02:43:37.176936 25803 raft_consensus.cc:388] T 00000000000000000000000000000000 P cc5c8fbb4cbc4f78919c978e4d2a7ce4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:43:37.176980 25803 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc5c8fbb4cbc4f78919c978e4d2a7ce4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:43:37.177026 25803 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc5c8fbb4cbc4f78919c978e4d2a7ce4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:37.177685 25803 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc5c8fbb4cbc4f78919c978e4d2a7ce4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc5c8fbb4cbc4f78919c978e4d2a7ce4" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45389 } }
I20260619 02:43:37.177805 25803 leader_election.cc:302] T 00000000000000000000000000000000 P cc5c8fbb4cbc4f78919c978e4d2a7ce4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cc5c8fbb4cbc4f78919c978e4d2a7ce4; no voters:
I20260619 02:43:37.177982 25803 leader_election.cc:288] T 00000000000000000000000000000000 P cc5c8fbb4cbc4f78919c978e4d2a7ce4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:43:37.178033 25808 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cc5c8fbb4cbc4f78919c978e4d2a7ce4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:43:37.178207 25808 raft_consensus.cc:686] T 00000000000000000000000000000000 P cc5c8fbb4cbc4f78919c978e4d2a7ce4 [term 1 LEADER]: Becoming Leader. State: Replica: cc5c8fbb4cbc4f78919c978e4d2a7ce4, State: Running, Role: LEADER
I20260619 02:43:37.178257 25803 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc5c8fbb4cbc4f78919c978e4d2a7ce4 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:43:37.178303 25808 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc5c8fbb4cbc4f78919c978e4d2a7ce4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc5c8fbb4cbc4f78919c978e4d2a7ce4" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45389 } }
I20260619 02:43:37.178884 25810 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc5c8fbb4cbc4f78919c978e4d2a7ce4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc5c8fbb4cbc4f78919c978e4d2a7ce4. Latest consensus state: current_term: 1 leader_uuid: "cc5c8fbb4cbc4f78919c978e4d2a7ce4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc5c8fbb4cbc4f78919c978e4d2a7ce4" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45389 } } }
I20260619 02:43:37.178972 25810 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc5c8fbb4cbc4f78919c978e4d2a7ce4 [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:37.179615 25806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:37.171921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32908 (local address 127.5.241.126:45389)
0619 02:43:37.172163 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:37.172172 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:37.172207 (+ 35us) server_negotiation.cc:407] Connection header received
0619 02:43:37.172718 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:37.172727 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:37.172806 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:37.172978 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:37.173215 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:37.174510 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:37.175403 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:37.176067 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:37.176222 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:37.178644 (+ 2422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:37.178648 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:37.178649 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:43:37.178909 (+ 260us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:37.178957 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:43:37.179090 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:37.179175 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:37.179301 (+ 126us) server_negotiation.cc:299] Negotiation successful
0619 02:43:37.179395 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":87,"threads_started":1}
I20260619 02:43:37.179718 25817 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:43:37.179394 25809 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc5c8fbb4cbc4f78919c978e4d2a7ce4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cc5c8fbb4cbc4f78919c978e4d2a7ce4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc5c8fbb4cbc4f78919c978e4d2a7ce4" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45389 } } }
I20260619 02:43:37.180040 25809 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc5c8fbb4cbc4f78919c978e4d2a7ce4 [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:37.180450 25817 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:43:37.181898 25817 catalog_manager.cc:1347] Generated new cluster ID: 9610b4d5c426412da13b3dbfac9e34ed
I20260619 02:43:37.181951 25817 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:43:37.200327 25817 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:43:37.201090 25817 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:43:37.211179 25817 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cc5c8fbb4cbc4f78919c978e4d2a7ce4: Generated new TSK 0
I20260619 02:43:37.211406 25817 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:43:37.215760 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.65:0
--local_ip_for_outbound_sockets=127.5.241.65
--webserver_interface=127.5.241.65
--webserver_port=0
--tserver_master_addrs=127.5.241.126:45389
--builtin_ntp_servers=127.5.241.84:44391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 02:43:37.297137 25827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:37.297386 25827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:37.297425 25827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:37.298616 25827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:37.298694 25827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:37.298736 25827 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.65
I20260619 02:43:37.300141 25827 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:44391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.241.65
--webserver_port=0
--tserver_master_addrs=127.5.241.126:45389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25827
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.65
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:43:37.300421 25827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:37.300740 25827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:37.305980 25833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:37.305967 25832 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:37.306073 25835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:37.306453 25827 server_base.cc:1034] running on GCE node
I20260619 02:43:37.306666 25827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:37.306972 25827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:37.308130 25827 hybrid_clock.cc:648] HybridClock initialized: now 1781837017308108 us; error 40 us; skew 500 ppm
I20260619 02:43:37.309433 25827 webserver.cc:460] Webserver started at http://127.5.241.65:43371/ using document root <none> and password file <none>
I20260619 02:43:37.309731 25827 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:37.309782 25827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:37.309934 25827 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:43:37.311091 25827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/ts-0/data/instance:
uuid: "aca2658163b246b88856cc2f9a40a4d3"
format_stamp: "Formatted at 2026-06-19 02:43:37 on dist-test-slave-zpbg"
server_key: "7a78cebdd0f693410a12cad3df15a5f4"
server_key_iv: "807d63e92b274e28cad5451510ef4f62"
server_key_version: "encryptionkey@0"
I20260619 02:43:37.311457 25827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/ts-0/wal/instance:
uuid: "aca2658163b246b88856cc2f9a40a4d3"
format_stamp: "Formatted at 2026-06-19 02:43:37 on dist-test-slave-zpbg"
server_key: "7a78cebdd0f693410a12cad3df15a5f4"
server_key_iv: "807d63e92b274e28cad5451510ef4f62"
server_key_version: "encryptionkey@0"
I20260619 02:43:37.312784 25827 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:43:37.313557 25841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:37.313712 25827 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:43:37.313820 25827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/ts-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/ts-0/wal
uuid: "aca2658163b246b88856cc2f9a40a4d3"
format_stamp: "Formatted at 2026-06-19 02:43:37 on dist-test-slave-zpbg"
server_key: "7a78cebdd0f693410a12cad3df15a5f4"
server_key_iv: "807d63e92b274e28cad5451510ef4f62"
server_key_version: "encryptionkey@0"
I20260619 02:43:37.313894 25827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:37.330195 25827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:37.330533 25827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:37.330711 25827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:37.330998 25827 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:43:37.331347 25827 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:43:37.331408 25827 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:37.331452 25827 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:43:37.331477 25827 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:37.337284 25827 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.65:46829
I20260619 02:43:37.337327 25954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.65:46829 every 8 connection(s)
I20260619 02:43:37.337666 25827 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/ts-0/data/info.pb
I20260619 02:43:37.340837 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 25827
I20260619 02:43:37.340967 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/ts-0/wal/instance
I20260619 02:43:37.341183 6085 external_mini_cluster.cc:1428] Setting key 5052e497fadcb96b2038e0f9f53f8fde
I20260619 02:43:37.342147 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.66:0
--local_ip_for_outbound_sockets=127.5.241.66
--webserver_interface=127.5.241.66
--webserver_port=0
--tserver_master_addrs=127.5.241.126:45389
--builtin_ntp_servers=127.5.241.84:44391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 02:43:37.344031 25806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:37.338390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.65:47421 (local address 127.5.241.126:45389)
0619 02:43:37.338457 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:37.338459 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:37.338707 (+ 248us) server_negotiation.cc:407] Connection header received
0619 02:43:37.339280 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:37.339282 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:37.339299 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:37.339366 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:37.340065 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:37.340593 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:37.342365 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:37.342582 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:37.342683 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:37.343695 (+ 1012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:37.343697 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:37.343697 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:43:37.343703 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:37.343719 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:43:37.343724 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:37.343780 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:37.343911 (+ 131us) server_negotiation.cc:299] Negotiation successful
0619 02:43:37.343939 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 02:43:37.344234 25957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:37.338385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45389 (local address 127.5.241.65:47421)
0619 02:43:37.338585 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:37.338619 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:37.339163 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:37.339441 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:37.339445 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:37.339453 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:43:37.339963 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:37.339967 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:37.341099 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:37.341100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:37.342200 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:37.342204 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:37.342696 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:37.342697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:37.342898 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:37.343523 (+ 625us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:43:37.343523 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:43:37.343552 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:43:37.343576 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:43:37.343792 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:43:37.343798 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:43:37.343800 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 02:43:37.343882 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 02:43:37.343992 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":93,"threads_started":1}
I20260619 02:43:37.344758 25955 heartbeater.cc:347] Connected to a master server at 127.5.241.126:45389
I20260619 02:43:37.344846 25955 heartbeater.cc:464] Registering TS with master...
I20260619 02:43:37.345093 25955 heartbeater.cc:511] Master 127.5.241.126:45389 requested a full tablet report, sending...
I20260619 02:43:37.345682 25767 ts_manager.cc:194] Registered new tserver with Master: aca2658163b246b88856cc2f9a40a4d3 (127.5.241.65:46829)
I20260619 02:43:37.346760 25767 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.241.65:47421
W20260619 02:43:37.424154 25958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:37.424383 25958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:37.424407 25958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:37.425793 25958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:37.425845 25958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:37.425874 25958 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.66
I20260619 02:43:37.427475 25958 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:44391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.241.66
--webserver_port=0
--tserver_master_addrs=127.5.241.126:45389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25958
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.66
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:43:37.427719 25958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:37.428056 25958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:37.433413 25964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:37.433427 25963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:37.433426 25966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:37.433655 25958 server_base.cc:1034] running on GCE node
I20260619 02:43:37.433954 25958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:37.434219 25958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:37.435391 25958 hybrid_clock.cc:648] HybridClock initialized: now 1781837017435370 us; error 41 us; skew 500 ppm
I20260619 02:43:37.436863 25958 webserver.cc:460] Webserver started at http://127.5.241.66:38313/ using document root <none> and password file <none>
I20260619 02:43:37.437157 25958 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:37.437242 25958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:37.437402 25958 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:43:37.438532 25958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/ts-1/data/instance:
uuid: "2cc87db568034628949f5a4d6cf6ad79"
format_stamp: "Formatted at 2026-06-19 02:43:37 on dist-test-slave-zpbg"
server_key: "deda22482335fe5f11609e8d9505368e"
server_key_iv: "cd86bdfb10bcfa6ac73f2bbf51468ce1"
server_key_version: "encryptionkey@0"
I20260619 02:43:37.438938 25958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/ts-1/wal/instance:
uuid: "2cc87db568034628949f5a4d6cf6ad79"
format_stamp: "Formatted at 2026-06-19 02:43:37 on dist-test-slave-zpbg"
server_key: "deda22482335fe5f11609e8d9505368e"
server_key_iv: "cd86bdfb10bcfa6ac73f2bbf51468ce1"
server_key_version: "encryptionkey@0"
I20260619 02:43:37.440315 25958 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:43:37.441184 25972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:37.441424 25958 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:43:37.441529 25958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/ts-1/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/ts-1/wal
uuid: "2cc87db568034628949f5a4d6cf6ad79"
format_stamp: "Formatted at 2026-06-19 02:43:37 on dist-test-slave-zpbg"
server_key: "deda22482335fe5f11609e8d9505368e"
server_key_iv: "cd86bdfb10bcfa6ac73f2bbf51468ce1"
server_key_version: "encryptionkey@0"
I20260619 02:43:37.441624 25958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:37.455471 25958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:37.455798 25958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:37.455948 25958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:37.456182 25958 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:43:37.456513 25958 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:43:37.456568 25958 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:37.456610 25958 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:43:37.456636 25958 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:37.462504 25958 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.66:40621
I20260619 02:43:37.462553 26085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.66:40621 every 8 connection(s)
I20260619 02:43:37.462946 25958 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/ts-1/data/info.pb
I20260619 02:43:37.467479 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 25958
I20260619 02:43:37.467602 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/ts-1/wal/instance
I20260619 02:43:37.467828 6085 external_mini_cluster.cc:1428] Setting key f4f00862091fd4753b4ab4a7bf2f1ca4
I20260619 02:43:37.468474 25806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:37.463679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.66:53975 (local address 127.5.241.126:45389)
0619 02:43:37.463752 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:37.463754 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:37.463962 (+ 208us) server_negotiation.cc:407] Connection header received
0619 02:43:37.464505 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:37.464507 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:37.464518 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:37.464583 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:37.465279 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:37.465786 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:37.466794 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:37.467013 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:37.467106 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:37.468160 (+ 1054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:37.468161 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:37.468162 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:43:37.468167 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:37.468183 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:43:37.468187 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:37.468241 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:37.468367 (+ 126us) server_negotiation.cc:299] Negotiation successful
0619 02:43:37.468397 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 02:43:37.468741 26088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:37.463643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45389 (local address 127.5.241.66:53975)
0619 02:43:37.463836 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:37.463873 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:37.464393 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:37.464663 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:37.464667 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:37.464677 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:43:37.465177 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:37.465181 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:37.465888 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:37.465889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:37.466686 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:37.466688 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:37.467120 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:37.467121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:37.467262 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:37.467916 (+ 654us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:43:37.467916 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:43:37.467989 (+ 73us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:43:37.468017 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:43:37.468255 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:43:37.468261 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:43:37.468264 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:43:37.468365 (+ 101us) client_negotiation.cc:240] Negotiation successful
0619 02:43:37.468467 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":70,"threads_started":1}
I20260619 02:43:37.468787 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.241.67:0
--local_ip_for_outbound_sockets=127.5.241.67
--webserver_interface=127.5.241.67
--webserver_port=0
--tserver_master_addrs=127.5.241.126:45389
--builtin_ntp_servers=127.5.241.84:44391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 02:43:37.469220 26086 heartbeater.cc:347] Connected to a master server at 127.5.241.126:45389
I20260619 02:43:37.469309 26086 heartbeater.cc:464] Registering TS with master...
I20260619 02:43:37.469501 26086 heartbeater.cc:511] Master 127.5.241.126:45389 requested a full tablet report, sending...
I20260619 02:43:37.469859 25767 ts_manager.cc:194] Registered new tserver with Master: 2cc87db568034628949f5a4d6cf6ad79 (127.5.241.66:40621)
I20260619 02:43:37.470203 25767 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.241.66:53975
W20260619 02:43:37.549589 26089 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:37.549803 26089 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:37.549824 26089 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:37.550962 26089 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:37.550999 26089 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:37.551024 26089 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.241.67
I20260619 02:43:37.552304 26089 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:44391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.241.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.241.67
--webserver_port=0
--tserver_master_addrs=127.5.241.126:45389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26089
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.241.67
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:43:37.552498 26089 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:37.552752 26089 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:37.557690 26095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:37.557683 26094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:37.557725 26097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:37.558480 26089 server_base.cc:1034] running on GCE node
I20260619 02:43:37.558736 26089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:37.559057 26089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:37.560218 26089 hybrid_clock.cc:648] HybridClock initialized: now 1781837017560195 us; error 36 us; skew 500 ppm
I20260619 02:43:37.561606 26089 webserver.cc:460] Webserver started at http://127.5.241.67:37323/ using document root <none> and password file <none>
I20260619 02:43:37.561900 26089 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:37.561949 26089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:37.562105 26089 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:43:37.563318 26089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/ts-2/data/instance:
uuid: "27daedb3475b46c089a5b18b142a574c"
format_stamp: "Formatted at 2026-06-19 02:43:37 on dist-test-slave-zpbg"
server_key: "f0286e396a65b1e6889c965acea03289"
server_key_iv: "60469622bb5dccef62b52a88d7cb297f"
server_key_version: "encryptionkey@0"
I20260619 02:43:37.563673 26089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/ts-2/wal/instance:
uuid: "27daedb3475b46c089a5b18b142a574c"
format_stamp: "Formatted at 2026-06-19 02:43:37 on dist-test-slave-zpbg"
server_key: "f0286e396a65b1e6889c965acea03289"
server_key_iv: "60469622bb5dccef62b52a88d7cb297f"
server_key_version: "encryptionkey@0"
I20260619 02:43:37.564955 26089 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:43:37.565672 26103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:37.565848 26089 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:43:37.565948 26089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/ts-2/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/ts-2/wal
uuid: "27daedb3475b46c089a5b18b142a574c"
format_stamp: "Formatted at 2026-06-19 02:43:37 on dist-test-slave-zpbg"
server_key: "f0286e396a65b1e6889c965acea03289"
server_key_iv: "60469622bb5dccef62b52a88d7cb297f"
server_key_version: "encryptionkey@0"
I20260619 02:43:37.566025 26089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:37.591601 26089 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:37.591954 26089 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:37.592116 26089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:37.592370 26089 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:43:37.592726 26089 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:43:37.592783 26089 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:37.592847 26089 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:43:37.592877 26089 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:37.598621 26089 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.67:38295
I20260619 02:43:37.598667 26216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.67:38295 every 8 connection(s)
I20260619 02:43:37.599051 26089 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/ts-2/data/info.pb
I20260619 02:43:37.603744 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 26089
I20260619 02:43:37.603883 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836772852275-6085-0/minicluster-data/ts-2/wal/instance
I20260619 02:43:37.604110 6085 external_mini_cluster.cc:1428] Setting key da024413404f9bcca2b6bc70e48a18a3
I20260619 02:43:37.604560 25806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:37.599845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:54693 (local address 127.5.241.126:45389)
0619 02:43:37.599936 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:37.599938 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:37.600138 (+ 200us) server_negotiation.cc:407] Connection header received
0619 02:43:37.600639 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:37.600640 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:37.600651 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:37.600712 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:37.601422 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:37.601919 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:37.603041 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:37.603242 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:37.603310 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:37.604180 (+ 870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:37.604182 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:37.604182 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:43:37.604187 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:37.604205 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:43:37.604210 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:37.604285 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:37.604446 (+ 161us) server_negotiation.cc:299] Negotiation successful
0619 02:43:37.604474 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 02:43:37.604770 26219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:37.599814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45389 (local address 127.5.241.67:54693)
0619 02:43:37.600000 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:37.600040 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:37.600527 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:37.600771 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:37.600774 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:37.600783 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:43:37.601289 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:37.601293 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:37.602033 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:37.602034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:37.602942 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:37.602944 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:37.603326 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:37.603326 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:37.603434 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:37.604038 (+ 604us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:43:37.604038 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:43:37.604066 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:43:37.604089 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:43:37.604300 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:43:37.604307 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:43:37.604310 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:43:37.604389 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 02:43:37.604525 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":86,"threads_started":1}
I20260619 02:43:37.605268 26217 heartbeater.cc:347] Connected to a master server at 127.5.241.126:45389
I20260619 02:43:37.605357 26217 heartbeater.cc:464] Registering TS with master...
I20260619 02:43:37.605494 26217 heartbeater.cc:511] Master 127.5.241.126:45389 requested a full tablet report, sending...
I20260619 02:43:37.605837 25767 ts_manager.cc:194] Registered new tserver with Master: 27daedb3475b46c089a5b18b142a574c (127.5.241.67:38295)
I20260619 02:43:37.606154 25767 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.241.67:54693
I20260619 02:43:37.616274 6085 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:43:37.620684 25806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:37.617881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32916 (local address 127.5.241.126:45389)
0619 02:43:37.617946 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:37.617948 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:37.618044 (+ 96us) server_negotiation.cc:407] Connection header received
0619 02:43:37.618080 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:37.618081 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:37.618091 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:37.618140 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:37.618270 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:37.618840 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:37.619546 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:37.619755 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:37.619810 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:37.620398 (+ 588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:37.620399 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:37.620399 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:43:37.620403 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:37.620416 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:43:37.620420 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:37.620469 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:37.620522 (+ 53us) server_negotiation.cc:299] Negotiation successful
0619 02:43:37.620543 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 02:43:37.622391 25767 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:32916:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:43:37.623981 25767 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:43:37.631884 26230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:37.626799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.67:38295 (local address 127.0.0.1:41790)
0619 02:43:37.627165 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:37.627174 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:37.627220 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:37.627476 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:37.627478 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:37.627480 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:43:37.627519 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:37.627520 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:37.628883 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:37.628884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:37.630524 (+ 1640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:37.630528 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:37.631566 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:37.631567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:37.631644 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:37.631659 (+ 15us) client_negotiation.cc:769] Sending connection context
0619 02:43:37.631725 (+ 66us) client_negotiation.cc:240] Negotiation successful
0619 02:43:37.631798 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":473,"thread_start_us":107,"threads_started":1}
I20260619 02:43:37.632114 26227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:37.626493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41790 (local address 127.5.241.67:38295)
0619 02:43:37.626927 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:37.626931 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:37.627283 (+ 352us) server_negotiation.cc:407] Connection header received
0619 02:43:37.627326 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:37.627328 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:37.627342 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:37.627410 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:43:37.627671 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:37.628787 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:37.630627 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:37.631375 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:37.631448 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:37.631535 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:37.631993 (+ 458us) server_negotiation.cc:299] Negotiation successful
0619 02:43:37.632035 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":170,"threads_started":1}
I20260619 02:43:37.632653 26225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:37.626388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.66:40621 (local address 127.0.0.1:54580)
0619 02:43:37.626921 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:37.626963 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:37.627036 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:37.627337 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:37.627340 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:37.627342 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:43:37.627421 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:37.627424 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:37.629484 (+ 2060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:37.629485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:37.631416 (+ 1931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:37.631419 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:37.632426 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:37.632427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:37.632483 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:37.632487 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:43:37.632542 (+ 55us) client_negotiation.cc:240] Negotiation successful
0619 02:43:37.632582 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":52,"threads_started":1}
I20260619 02:43:37.632678 26228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:37.626615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54580 (local address 127.5.241.66:40621)
0619 02:43:37.626840 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:37.626847 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:37.627025 (+ 178us) server_negotiation.cc:407] Connection header received
0619 02:43:37.627152 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:37.627154 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:37.627174 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:37.627528 (+ 354us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:43:37.627944 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:37.629361 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:37.631512 (+ 2151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:37.632318 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:37.632422 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:37.632507 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:37.632558 (+ 51us) server_negotiation.cc:299] Negotiation successful
0619 02:43:37.632595 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":79,"threads_started":1}
I20260619 02:43:37.633160 26151 tablet_service.cc:1467] Processing CreateTablet for tablet 6cde2d91f8794b5caba5fe9230adf60e (DEFAULT_TABLE table=test-table [id=56816d8225054d64adc498355a2fb392]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:43:37.633353 26020 tablet_service.cc:1467] Processing CreateTablet for tablet 6cde2d91f8794b5caba5fe9230adf60e (DEFAULT_TABLE table=test-table [id=56816d8225054d64adc498355a2fb392]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:43:37.633558 26151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6cde2d91f8794b5caba5fe9230adf60e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:37.633689 26020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6cde2d91f8794b5caba5fe9230adf60e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:37.633904 26226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:37.626396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:46829 (local address 127.0.0.1:60252)
0619 02:43:37.626985 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:37.626996 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:37.627057 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:37.627342 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:37.627343 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:37.627346 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:43:37.627420 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:37.627422 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:37.628840 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:37.628841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:37.630483 (+ 1642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:37.630485 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:37.631568 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:37.631569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:37.631651 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:37.631659 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 02:43:37.631731 (+ 72us) client_negotiation.cc:240] Negotiation successful
0619 02:43:37.633826 (+ 2095us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"mutex_wait_us":2046,"thread_start_us":94,"threads_started":1}
I20260619 02:43:37.631918 26229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:37.626643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60252 (local address 127.5.241.65:46829)
0619 02:43:37.626840 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:37.626845 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:37.627134 (+ 289us) server_negotiation.cc:407] Connection header received
0619 02:43:37.627180 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:37.627183 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:37.627199 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:37.627273 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:43:37.627500 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:37.628739 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:37.630607 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:37.631353 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:37.631424 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:37.631517 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:37.631777 (+ 260us) server_negotiation.cc:299] Negotiation successful
0619 02:43:37.631817 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":75,"threads_started":1}
I20260619 02:43:37.635452 25889 tablet_service.cc:1467] Processing CreateTablet for tablet 6cde2d91f8794b5caba5fe9230adf60e (DEFAULT_TABLE table=test-table [id=56816d8225054d64adc498355a2fb392]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:43:37.635793 25889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6cde2d91f8794b5caba5fe9230adf60e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:37.636888 26231 tablet_bootstrap.cc:492] T 6cde2d91f8794b5caba5fe9230adf60e P 2cc87db568034628949f5a4d6cf6ad79: Bootstrap starting.
I20260619 02:43:37.637578 26231 tablet_bootstrap.cc:654] T 6cde2d91f8794b5caba5fe9230adf60e P 2cc87db568034628949f5a4d6cf6ad79: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:37.637876 26231 log.cc:824] T 6cde2d91f8794b5caba5fe9230adf60e P 2cc87db568034628949f5a4d6cf6ad79: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:37.638233 26232 tablet_bootstrap.cc:492] T 6cde2d91f8794b5caba5fe9230adf60e P 27daedb3475b46c089a5b18b142a574c: Bootstrap starting.
I20260619 02:43:37.638713 26231 tablet_bootstrap.cc:492] T 6cde2d91f8794b5caba5fe9230adf60e P 2cc87db568034628949f5a4d6cf6ad79: No bootstrap required, opened a new log
I20260619 02:43:37.638828 26231 ts_tablet_manager.cc:1397] T 6cde2d91f8794b5caba5fe9230adf60e P 2cc87db568034628949f5a4d6cf6ad79: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:43:37.638842 26232 tablet_bootstrap.cc:654] T 6cde2d91f8794b5caba5fe9230adf60e P 27daedb3475b46c089a5b18b142a574c: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:37.639050 26234 tablet_bootstrap.cc:492] T 6cde2d91f8794b5caba5fe9230adf60e P aca2658163b246b88856cc2f9a40a4d3: Bootstrap starting.
I20260619 02:43:37.639168 26232 log.cc:824] T 6cde2d91f8794b5caba5fe9230adf60e P 27daedb3475b46c089a5b18b142a574c: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:37.639752 26234 tablet_bootstrap.cc:654] T 6cde2d91f8794b5caba5fe9230adf60e P aca2658163b246b88856cc2f9a40a4d3: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:37.640033 26232 tablet_bootstrap.cc:492] T 6cde2d91f8794b5caba5fe9230adf60e P 27daedb3475b46c089a5b18b142a574c: No bootstrap required, opened a new log
I20260619 02:43:37.640061 26234 log.cc:824] T 6cde2d91f8794b5caba5fe9230adf60e P aca2658163b246b88856cc2f9a40a4d3: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:37.640105 26232 ts_tablet_manager.cc:1397] T 6cde2d91f8794b5caba5fe9230adf60e P 27daedb3475b46c089a5b18b142a574c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:43:37.640888 26234 tablet_bootstrap.cc:492] T 6cde2d91f8794b5caba5fe9230adf60e P aca2658163b246b88856cc2f9a40a4d3: No bootstrap required, opened a new log
I20260619 02:43:37.640972 26234 ts_tablet_manager.cc:1397] T 6cde2d91f8794b5caba5fe9230adf60e P aca2658163b246b88856cc2f9a40a4d3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:43:37.641361 26231 raft_consensus.cc:348] T 6cde2d91f8794b5caba5fe9230adf60e P 2cc87db568034628949f5a4d6cf6ad79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27daedb3475b46c089a5b18b142a574c" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38295 } } peers { permanent_uuid: "aca2658163b246b88856cc2f9a40a4d3" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46829 } } peers { permanent_uuid: "2cc87db568034628949f5a4d6cf6ad79" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 40621 } }
I20260619 02:43:37.641516 26231 raft_consensus.cc:374] T 6cde2d91f8794b5caba5fe9230adf60e P 2cc87db568034628949f5a4d6cf6ad79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:37.641559 26231 raft_consensus.cc:729] T 6cde2d91f8794b5caba5fe9230adf60e P 2cc87db568034628949f5a4d6cf6ad79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cc87db568034628949f5a4d6cf6ad79, State: Initialized, Role: FOLLOWER
I20260619 02:43:37.641708 26231 consensus_queue.cc:260] T 6cde2d91f8794b5caba5fe9230adf60e P 2cc87db568034628949f5a4d6cf6ad79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27daedb3475b46c089a5b18b142a574c" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38295 } } peers { permanent_uuid: "aca2658163b246b88856cc2f9a40a4d3" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46829 } } peers { permanent_uuid: "2cc87db568034628949f5a4d6cf6ad79" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 40621 } }
I20260619 02:43:37.641971 26231 ts_tablet_manager.cc:1428] T 6cde2d91f8794b5caba5fe9230adf60e P 2cc87db568034628949f5a4d6cf6ad79: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 02:43:37.642099 26086 heartbeater.cc:503] Master 127.5.241.126:45389 was elected leader, sending a full tablet report...
I20260619 02:43:37.642709 26232 raft_consensus.cc:348] T 6cde2d91f8794b5caba5fe9230adf60e P 27daedb3475b46c089a5b18b142a574c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27daedb3475b46c089a5b18b142a574c" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38295 } } peers { permanent_uuid: "aca2658163b246b88856cc2f9a40a4d3" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46829 } } peers { permanent_uuid: "2cc87db568034628949f5a4d6cf6ad79" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 40621 } }
I20260619 02:43:37.642901 26232 raft_consensus.cc:374] T 6cde2d91f8794b5caba5fe9230adf60e P 27daedb3475b46c089a5b18b142a574c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:37.642944 26232 raft_consensus.cc:729] T 6cde2d91f8794b5caba5fe9230adf60e P 27daedb3475b46c089a5b18b142a574c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27daedb3475b46c089a5b18b142a574c, State: Initialized, Role: FOLLOWER
I20260619 02:43:37.643054 26232 consensus_queue.cc:260] T 6cde2d91f8794b5caba5fe9230adf60e P 27daedb3475b46c089a5b18b142a574c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27daedb3475b46c089a5b18b142a574c" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38295 } } peers { permanent_uuid: "aca2658163b246b88856cc2f9a40a4d3" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46829 } } peers { permanent_uuid: "2cc87db568034628949f5a4d6cf6ad79" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 40621 } }
I20260619 02:43:37.643316 26232 ts_tablet_manager.cc:1428] T 6cde2d91f8794b5caba5fe9230adf60e P 27daedb3475b46c089a5b18b142a574c: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260619 02:43:37.643438 26217 heartbeater.cc:503] Master 127.5.241.126:45389 was elected leader, sending a full tablet report...
I20260619 02:43:37.643508 26234 raft_consensus.cc:348] T 6cde2d91f8794b5caba5fe9230adf60e P aca2658163b246b88856cc2f9a40a4d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27daedb3475b46c089a5b18b142a574c" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38295 } } peers { permanent_uuid: "aca2658163b246b88856cc2f9a40a4d3" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46829 } } peers { permanent_uuid: "2cc87db568034628949f5a4d6cf6ad79" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 40621 } }
I20260619 02:43:37.643707 26234 raft_consensus.cc:374] T 6cde2d91f8794b5caba5fe9230adf60e P aca2658163b246b88856cc2f9a40a4d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:37.643743 26234 raft_consensus.cc:729] T 6cde2d91f8794b5caba5fe9230adf60e P aca2658163b246b88856cc2f9a40a4d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aca2658163b246b88856cc2f9a40a4d3, State: Initialized, Role: FOLLOWER
I20260619 02:43:37.643878 26234 consensus_queue.cc:260] T 6cde2d91f8794b5caba5fe9230adf60e P aca2658163b246b88856cc2f9a40a4d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27daedb3475b46c089a5b18b142a574c" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38295 } } peers { permanent_uuid: "aca2658163b246b88856cc2f9a40a4d3" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46829 } } peers { permanent_uuid: "2cc87db568034628949f5a4d6cf6ad79" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 40621 } }
I20260619 02:43:37.644166 26234 ts_tablet_manager.cc:1428] T 6cde2d91f8794b5caba5fe9230adf60e P aca2658163b246b88856cc2f9a40a4d3: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260619 02:43:37.644212 25955 heartbeater.cc:503] Master 127.5.241.126:45389 was elected leader, sending a full tablet report...
W20260619 02:43:37.713486 26087 tablet.cc:2369] T 6cde2d91f8794b5caba5fe9230adf60e P 2cc87db568034628949f5a4d6cf6ad79: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:43:37.838351 25956 tablet.cc:2369] T 6cde2d91f8794b5caba5fe9230adf60e P aca2658163b246b88856cc2f9a40a4d3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:43:37.849615 26218 tablet.cc:2369] T 6cde2d91f8794b5caba5fe9230adf60e P 27daedb3475b46c089a5b18b142a574c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:43:37.889230 26238 raft_consensus.cc:482] T 6cde2d91f8794b5caba5fe9230adf60e P 27daedb3475b46c089a5b18b142a574c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:43:37.889370 26238 raft_consensus.cc:504] T 6cde2d91f8794b5caba5fe9230adf60e P 27daedb3475b46c089a5b18b142a574c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27daedb3475b46c089a5b18b142a574c" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38295 } } peers { permanent_uuid: "aca2658163b246b88856cc2f9a40a4d3" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46829 } } peers { permanent_uuid: "2cc87db568034628949f5a4d6cf6ad79" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 40621 } }
I20260619 02:43:37.889811 26238 leader_election.cc:288] T 6cde2d91f8794b5caba5fe9230adf60e P 27daedb3475b46c089a5b18b142a574c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aca2658163b246b88856cc2f9a40a4d3 (127.5.241.65:46829), 2cc87db568034628949f5a4d6cf6ad79 (127.5.241.66:40621)
I20260619 02:43:37.893929 26228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:37.890190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:47441 (local address 127.5.241.66:40621)
0619 02:43:37.890232 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:37.890234 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:37.890242 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:43:37.890277 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:37.890278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:37.890287 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:37.890381 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:43:37.890614 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:37.891379 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:37.892975 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:37.893518 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:37.893598 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:37.893606 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:37.893811 (+ 205us) server_negotiation.cc:299] Negotiation successful
0619 02:43:37.893850 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 02:43:37.893922 26229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:37.890155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.241.67:42141 (local address 127.5.241.65:46829)
0619 02:43:37.890205 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:37.890208 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:37.890356 (+ 148us) server_negotiation.cc:407] Connection header received
0619 02:43:37.890383 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:37.890384 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:37.890397 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:37.890451 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:43:37.890600 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:37.891383 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:37.892975 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:37.893518 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:37.893598 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:37.893606 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:37.893811 (+ 205us) server_negotiation.cc:299] Negotiation successful
0619 02:43:37.893850 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 02:43:37.893940 26219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:37.890058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.66:40621 (local address 127.5.241.67:47441)
0619 02:43:37.890126 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:37.890139 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:37.890189 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:37.890411 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:37.890412 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:37.890414 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:43:37.890511 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:37.890513 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:37.891502 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:37.891503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:37.892864 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:37.892867 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:37.893638 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:37.893639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:37.893717 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:37.893722 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 02:43:37.893785 (+ 63us) client_negotiation.cc:240] Negotiation successful
0619 02:43:37.893842 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":81}
I20260619 02:43:37.893945 26240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:37.890059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:46829 (local address 127.5.241.67:42141)
0619 02:43:37.890249 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:37.890258 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:37.890318 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:37.890467 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:37.890467 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:37.890469 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:43:37.890524 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:37.890525 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:37.891502 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:37.891503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:37.892864 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:37.892867 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:37.893657 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:37.893658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:37.893725 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:37.893727 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 02:43:37.893785 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 02:43:37.893842 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":114,"thread_start_us":101,"threads_started":1}
I20260619 02:43:37.894368 26040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6cde2d91f8794b5caba5fe9230adf60e" candidate_uuid: "27daedb3475b46c089a5b18b142a574c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cc87db568034628949f5a4d6cf6ad79" is_pre_election: true
I20260619 02:43:37.894389 25909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6cde2d91f8794b5caba5fe9230adf60e" candidate_uuid: "27daedb3475b46c089a5b18b142a574c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aca2658163b246b88856cc2f9a40a4d3" is_pre_election: true
I20260619 02:43:37.894542 26040 raft_consensus.cc:2454] T 6cde2d91f8794b5caba5fe9230adf60e P 2cc87db568034628949f5a4d6cf6ad79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27daedb3475b46c089a5b18b142a574c in term 0.
I20260619 02:43:37.894542 25909 raft_consensus.cc:2454] T 6cde2d91f8794b5caba5fe9230adf60e P aca2658163b246b88856cc2f9a40a4d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27daedb3475b46c089a5b18b142a574c in term 0.
I20260619 02:43:37.894843 26106 leader_election.cc:302] T 6cde2d91f8794b5caba5fe9230adf60e P 27daedb3475b46c089a5b18b142a574c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27daedb3475b46c089a5b18b142a574c, aca2658163b246b88856cc2f9a40a4d3; no voters:
I20260619 02:43:37.894984 26238 raft_consensus.cc:2781] T 6cde2d91f8794b5caba5fe9230adf60e P 27daedb3475b46c089a5b18b142a574c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:43:37.895044 26238 raft_consensus.cc:482] T 6cde2d91f8794b5caba5fe9230adf60e P 27daedb3475b46c089a5b18b142a574c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:43:37.895062 26238 raft_consensus.cc:3037] T 6cde2d91f8794b5caba5fe9230adf60e P 27daedb3475b46c089a5b18b142a574c [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:37.895736 26238 raft_consensus.cc:504] T 6cde2d91f8794b5caba5fe9230adf60e P 27daedb3475b46c089a5b18b142a574c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27daedb3475b46c089a5b18b142a574c" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38295 } } peers { permanent_uuid: "aca2658163b246b88856cc2f9a40a4d3" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46829 } } peers { permanent_uuid: "2cc87db568034628949f5a4d6cf6ad79" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 40621 } }
I20260619 02:43:37.895885 26238 leader_election.cc:288] T 6cde2d91f8794b5caba5fe9230adf60e P 27daedb3475b46c089a5b18b142a574c [CANDIDATE]: Term 1 election: Requested vote from peers aca2658163b246b88856cc2f9a40a4d3 (127.5.241.65:46829), 2cc87db568034628949f5a4d6cf6ad79 (127.5.241.66:40621)
I20260619 02:43:37.896032 25909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6cde2d91f8794b5caba5fe9230adf60e" candidate_uuid: "27daedb3475b46c089a5b18b142a574c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aca2658163b246b88856cc2f9a40a4d3"
I20260619 02:43:37.896068 26040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6cde2d91f8794b5caba5fe9230adf60e" candidate_uuid: "27daedb3475b46c089a5b18b142a574c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cc87db568034628949f5a4d6cf6ad79"
I20260619 02:43:37.896147 25909 raft_consensus.cc:3037] T 6cde2d91f8794b5caba5fe9230adf60e P aca2658163b246b88856cc2f9a40a4d3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:37.896143 26040 raft_consensus.cc:3037] T 6cde2d91f8794b5caba5fe9230adf60e P 2cc87db568034628949f5a4d6cf6ad79 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:37.896845 25909 raft_consensus.cc:2454] T 6cde2d91f8794b5caba5fe9230adf60e P aca2658163b246b88856cc2f9a40a4d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27daedb3475b46c089a5b18b142a574c in term 1.
I20260619 02:43:37.896914 26040 raft_consensus.cc:2454] T 6cde2d91f8794b5caba5fe9230adf60e P 2cc87db568034628949f5a4d6cf6ad79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27daedb3475b46c089a5b18b142a574c in term 1.
I20260619 02:43:37.897068 26106 leader_election.cc:302] T 6cde2d91f8794b5caba5fe9230adf60e P 27daedb3475b46c089a5b18b142a574c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27daedb3475b46c089a5b18b142a574c, aca2658163b246b88856cc2f9a40a4d3; no voters:
I20260619 02:43:37.897158 26238 raft_consensus.cc:2781] T 6cde2d91f8794b5caba5fe9230adf60e P 27daedb3475b46c089a5b18b142a574c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:43:37.897311 26238 raft_consensus.cc:686] T 6cde2d91f8794b5caba5fe9230adf60e P 27daedb3475b46c089a5b18b142a574c [term 1 LEADER]: Becoming Leader. State: Replica: 27daedb3475b46c089a5b18b142a574c, State: Running, Role: LEADER
I20260619 02:43:37.897404 26238 consensus_queue.cc:237] T 6cde2d91f8794b5caba5fe9230adf60e P 27daedb3475b46c089a5b18b142a574c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27daedb3475b46c089a5b18b142a574c" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38295 } } peers { permanent_uuid: "aca2658163b246b88856cc2f9a40a4d3" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46829 } } peers { permanent_uuid: "2cc87db568034628949f5a4d6cf6ad79" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 40621 } }
I20260619 02:43:37.898263 25767 catalog_manager.cc:5495] T 6cde2d91f8794b5caba5fe9230adf60e P 27daedb3475b46c089a5b18b142a574c reported cstate change: term changed from 0 to 1, leader changed from <none> to 27daedb3475b46c089a5b18b142a574c (127.5.241.67). New cstate: current_term: 1 leader_uuid: "27daedb3475b46c089a5b18b142a574c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27daedb3475b46c089a5b18b142a574c" member_type: VOTER last_known_addr { host: "127.5.241.67" port: 38295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aca2658163b246b88856cc2f9a40a4d3" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2cc87db568034628949f5a4d6cf6ad79" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 40621 } health_report { overall_health: UNKNOWN } } }
I20260619 02:43:37.918402 26227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:37.915705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41798 (local address 127.5.241.67:38295)
0619 02:43:37.915774 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:37.915776 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:37.915804 (+ 28us) server_negotiation.cc:407] Connection header received
0619 02:43:37.915850 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:37.915851 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:37.915869 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:37.915957 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:43:37.916133 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:37.916605 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:37.917468 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:37.917670 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:37.917745 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:37.917946 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:43:37.918049 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:43:37.918150 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:37.918258 (+ 108us) server_negotiation.cc:299] Negotiation successful
0619 02:43:37.918277 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 02:43:37.920356 25909 raft_consensus.cc:1261] T 6cde2d91f8794b5caba5fe9230adf60e P aca2658163b246b88856cc2f9a40a4d3 [term 1 FOLLOWER]: Refusing update from remote peer 27daedb3475b46c089a5b18b142a574c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:43:37.920356 26040 raft_consensus.cc:1261] T 6cde2d91f8794b5caba5fe9230adf60e P 2cc87db568034628949f5a4d6cf6ad79 [term 1 FOLLOWER]: Refusing update from remote peer 27daedb3475b46c089a5b18b142a574c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:43:37.920816 26241 consensus_queue.cc:1035] T 6cde2d91f8794b5caba5fe9230adf60e P 27daedb3475b46c089a5b18b142a574c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cc87db568034628949f5a4d6cf6ad79" member_type: VOTER last_known_addr { host: "127.5.241.66" port: 40621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:43:37.921003 26238 consensus_queue.cc:1035] T 6cde2d91f8794b5caba5fe9230adf60e P 27daedb3475b46c089a5b18b142a574c [LEADER]: Connected to new peer: Peer: permanent_uuid: "aca2658163b246b88856cc2f9a40a4d3" member_type: VOTER last_known_addr { host: "127.5.241.65" port: 46829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:43:37.923416 26244 mvcc.cc:205] Tried to move back new op lower bound from 7298404425398628352 to 7298404425308258304. Current Snapshot: MvccSnapshot[applied={T|T < 7298404425398628352}]
I20260619 02:43:37.923733 26245 mvcc.cc:205] Tried to move back new op lower bound from 7298404425398628352 to 7298404425308258304. Current Snapshot: MvccSnapshot[applied={T|T < 7298404425398628352}]
I20260619 02:43:37.924163 26246 mvcc.cc:205] Tried to move back new op lower bound from 7298404425398628352 to 7298404425308258304. Current Snapshot: MvccSnapshot[applied={T|T < 7298404425398628352}]
I20260619 02:43:37.926623 25767 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:32916:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:43:37.926733 25767 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:32916:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:43:37.927518 25767 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cc5c8fbb4cbc4f78919c978e4d2a7ce4: Sending DeleteTablet for 3 replicas of tablet 6cde2d91f8794b5caba5fe9230adf60e
I20260619 02:43:37.927824 26151 tablet_service.cc:1514] Processing DeleteTablet for tablet 6cde2d91f8794b5caba5fe9230adf60e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:43:37 UTC) from {username='slave'} at 127.0.0.1:41790
I20260619 02:43:37.927883 25889 tablet_service.cc:1514] Processing DeleteTablet for tablet 6cde2d91f8794b5caba5fe9230adf60e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:43:37 UTC) from {username='slave'} at 127.0.0.1:60252
I20260619 02:43:37.928182 26254 tablet_replica.cc:330] T 6cde2d91f8794b5caba5fe9230adf60e P aca2658163b246b88856cc2f9a40a4d3: stopping tablet replica
I20260619 02:43:37.928185 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 25827
I20260619 02:43:37.928292 26254 raft_consensus.cc:2229] T 6cde2d91f8794b5caba5fe9230adf60e P aca2658163b246b88856cc2f9a40a4d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:43:37.928406 26254 raft_consensus.cc:2258] T 6cde2d91f8794b5caba5fe9230adf60e P aca2658163b246b88856cc2f9a40a4d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:43:37.928788 26020 tablet_service.cc:1514] Processing DeleteTablet for tablet 6cde2d91f8794b5caba5fe9230adf60e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:43:37 UTC) from {username='slave'} at 127.0.0.1:54580
I20260619 02:43:37.929811 26253 tablet_replica.cc:330] T 6cde2d91f8794b5caba5fe9230adf60e P 27daedb3475b46c089a5b18b142a574c: stopping tablet replica
I20260619 02:43:37.929915 26253 raft_consensus.cc:2229] T 6cde2d91f8794b5caba5fe9230adf60e P 27daedb3475b46c089a5b18b142a574c [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:43:37.930049 26253 raft_consensus.cc:2258] T 6cde2d91f8794b5caba5fe9230adf60e P 27daedb3475b46c089a5b18b142a574c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260619 02:43:37.933796 25753 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.241.65:46829 (error 108)
I20260619 02:43:37.933987 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 25958
I20260619 02:43:37.934919 26226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:37.934631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.65:46829 (local address 127.0.0.1:60268)
0619 02:43:37.934823 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:37.934876 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.65:46829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167}
W20260619 02:43:37.935055 25753 catalog_manager.cc:4584] TS aca2658163b246b88856cc2f9a40a4d3 (127.5.241.65:46829): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6cde2d91f8794b5caba5fe9230adf60e: Network error: Client connection negotiation failed: client connection to 127.5.241.65:46829: connect: Connection refused (error 111)
I20260619 02:43:37.937970 26253 ts_tablet_manager.cc:1905] T 6cde2d91f8794b5caba5fe9230adf60e P 27daedb3475b46c089a5b18b142a574c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:43:37.939438 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 26089
I20260619 02:43:37.939467 26226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:37.939357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.66:40621 (local address 127.0.0.1:54596)
0619 02:43:37.939405 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:37.939431 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.66:40621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260619 02:43:37.944235 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 25736
I20260619 02:43:37.944263 26226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:37.944028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.67:38295 (local address 127.0.0.1:41806)
0619 02:43:37.944186 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:37.944220 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.67:38295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139}
2026-06-19T02:43:37Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (945 ms)
[----------] 6 tests from EncryptionPolicyTest (8820 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836772852275-6085-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:43:38 dist-test-slave-zpbg krb5kdc[26259](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:43:38 dist-test-slave-zpbg krb5kdc[26259](info): set up 2 sockets
Jun 19 02:43:38 dist-test-slave-zpbg krb5kdc[26259](info): commencing operation
krb5kdc: starting...
W20260619 02:43:38.990615 6085 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Jun 19 02:43:39 dist-test-slave-zpbg krb5kdc[26259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837019, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:43:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:43:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836772852275-6085-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836772852275-6085-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:43:39.109490 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46177
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836772852275-6085-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46177
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:43:39.189149 26272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:39.189411 26272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:39.189453 26272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:39.190623 26272 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:43:39.190701 26272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:39.190727 26272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:39.190760 26272 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:43:39.190804 26272 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:43:39.192269 26272 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836772852275-6085-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46177
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836772852275-6085-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46177
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26272
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:43:39.192549 26272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:39.192778 26272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:39.197692 26277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:39.197865 26272 server_base.cc:1034] running on GCE node
W20260619 02:43:39.197710 26278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:39.197729 26280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:39.198167 26272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:39.198462 26272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:39.199633 26272 hybrid_clock.cc:648] HybridClock initialized: now 1781837019199618 us; error 28 us; skew 500 ppm
Jun 19 02:43:39 dist-test-slave-zpbg krb5kdc[26259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837019, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:39.202037 26272 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:43:39.202507 26272 webserver.cc:460] Webserver started at http://127.0.0.1:35825/ using document root <none> and password file <none>
I20260619 02:43:39.202739 26272 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:39.202828 26272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:39.202986 26272 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:43:39.204042 26272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836772852275-6085-0/minicluster-data/master-0/data/instance:
uuid: "2a456fd94c0a4d949653fce0dbd8a338"
format_stamp: "Formatted at 2026-06-19 02:43:39 on dist-test-slave-zpbg"
server_key: "7afee4d65bcb107334f98dbe884b6110"
server_key_iv: "d640503856180d62f1c08ec6eab18035"
server_key_version: "encryptionkey@0"
I20260619 02:43:39.204412 26272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836772852275-6085-0/minicluster-data/master-0/wal/instance:
uuid: "2a456fd94c0a4d949653fce0dbd8a338"
format_stamp: "Formatted at 2026-06-19 02:43:39 on dist-test-slave-zpbg"
server_key: "7afee4d65bcb107334f98dbe884b6110"
server_key_iv: "d640503856180d62f1c08ec6eab18035"
server_key_version: "encryptionkey@0"
I20260619 02:43:39.205729 26272 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:43:39.206436 26287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:39.206611 26272 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:43:39.206725 26272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "2a456fd94c0a4d949653fce0dbd8a338"
format_stamp: "Formatted at 2026-06-19 02:43:39 on dist-test-slave-zpbg"
server_key: "7afee4d65bcb107334f98dbe884b6110"
server_key_iv: "d640503856180d62f1c08ec6eab18035"
server_key_version: "encryptionkey@0"
I20260619 02:43:39.206835 26272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:39.230067 26272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:39.232527 26272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:39.232722 26272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:39.236810 26272 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46177
I20260619 02:43:39.236866 26339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46177 every 8 connection(s)
I20260619 02:43:39.237248 26272 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:43:39.238224 26340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:39.240564 26340 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a456fd94c0a4d949653fce0dbd8a338: Bootstrap starting.
I20260619 02:43:39.241160 26340 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a456fd94c0a4d949653fce0dbd8a338: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:39.241449 26340 log.cc:824] T 00000000000000000000000000000000 P 2a456fd94c0a4d949653fce0dbd8a338: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:39.242225 26340 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a456fd94c0a4d949653fce0dbd8a338: No bootstrap required, opened a new log
I20260619 02:43:39.244185 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 26272
I20260619 02:43:39.244117 26340 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a456fd94c0a4d949653fce0dbd8a338 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a456fd94c0a4d949653fce0dbd8a338" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46177 } }
I20260619 02:43:39.244308 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836772852275-6085-0/minicluster-data/master-0/wal/instance
I20260619 02:43:39.244341 26340 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a456fd94c0a4d949653fce0dbd8a338 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:39.244386 26340 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a456fd94c0a4d949653fce0dbd8a338 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a456fd94c0a4d949653fce0dbd8a338, State: Initialized, Role: FOLLOWER
I20260619 02:43:39.244526 6085 external_mini_cluster.cc:1428] Setting key 50d4cefc71e13a591ed3a794a2614b3a
I20260619 02:43:39.244553 26340 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a456fd94c0a4d949653fce0dbd8a338 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a456fd94c0a4d949653fce0dbd8a338" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46177 } }
I20260619 02:43:39.244644 26340 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2a456fd94c0a4d949653fce0dbd8a338 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:43:39.244689 26340 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a456fd94c0a4d949653fce0dbd8a338 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:43:39.244746 26340 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a456fd94c0a4d949653fce0dbd8a338 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:39.245937 26340 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a456fd94c0a4d949653fce0dbd8a338 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a456fd94c0a4d949653fce0dbd8a338" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46177 } }
I20260619 02:43:39.246066 26340 leader_election.cc:302] T 00000000000000000000000000000000 P 2a456fd94c0a4d949653fce0dbd8a338 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a456fd94c0a4d949653fce0dbd8a338; no voters:
I20260619 02:43:39.246258 26340 leader_election.cc:288] T 00000000000000000000000000000000 P 2a456fd94c0a4d949653fce0dbd8a338 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:43:39.246299 26345 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a456fd94c0a4d949653fce0dbd8a338 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:43:39.246443 26345 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a456fd94c0a4d949653fce0dbd8a338 [term 1 LEADER]: Becoming Leader. State: Replica: 2a456fd94c0a4d949653fce0dbd8a338, State: Running, Role: LEADER
I20260619 02:43:39.246532 26340 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a456fd94c0a4d949653fce0dbd8a338 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:43:39.246539 26345 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a456fd94c0a4d949653fce0dbd8a338 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a456fd94c0a4d949653fce0dbd8a338" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46177 } }
I20260619 02:43:39.246932 26346 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a456fd94c0a4d949653fce0dbd8a338 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2a456fd94c0a4d949653fce0dbd8a338" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a456fd94c0a4d949653fce0dbd8a338" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46177 } } }
I20260619 02:43:39.247016 26346 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a456fd94c0a4d949653fce0dbd8a338 [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:39.247164 26347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a456fd94c0a4d949653fce0dbd8a338 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a456fd94c0a4d949653fce0dbd8a338. Latest consensus state: current_term: 1 leader_uuid: "2a456fd94c0a4d949653fce0dbd8a338" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a456fd94c0a4d949653fce0dbd8a338" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46177 } } }
I20260619 02:43:39.247422 26347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a456fd94c0a4d949653fce0dbd8a338 [sys.catalog]: This master's current role is: LEADER
W20260619 02:43:39.248401 26361 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2a456fd94c0a4d949653fce0dbd8a338: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:43:39.248468 26361 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:43:39.248518 26351 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:43:39.248664 26351 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:43:39.251101 26351 catalog_manager.cc:1347] Generated new cluster ID: 568fa82339884db79e1f14447a1950e9
I20260619 02:43:39.251163 26351 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 02:43:39 dist-test-slave-zpbg krb5kdc[26259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837019, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 02:43:39.257787 26344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:39.245052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41621 (local address 127.0.0.1:46177)
0619 02:43:39.245306 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:39.245320 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:39.245358 (+ 38us) server_negotiation.cc:407] Connection header received
0619 02:43:39.245519 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:39.245529 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:39.245605 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:39.245778 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:39.246457 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:39.248616 (+ 2159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:39.249696 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:39.250337 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:39.250464 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:39.253458 (+ 2994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:39.253462 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:39.253463 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:39.253474 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:39.256004 (+ 2530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:39.256504 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:39.256507 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:39.256507 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:39.256617 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:39.256880 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:39.256881 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:39.256881 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:39.257184 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:39.257332 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:39.257488 (+ 156us) server_negotiation.cc:299] Negotiation successful
0619 02:43:39.257584 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":117,"threads_started":1}
I20260619 02:43:39.259055 6085 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:43:39.267980 26351 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:43:39.268241 26344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:39.260550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33203 (local address 127.0.0.1:46177)
0619 02:43:39.260590 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:39.260592 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:39.260602 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:43:39.260656 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:39.260657 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:39.260667 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:39.260735 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:39.261292 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:39.262151 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:39.263139 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:39.263525 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:39.263604 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:39.264964 (+ 1360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:39.264966 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:39.264966 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:39.264974 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:39.266656 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:39.267090 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:39.267091 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:39.267092 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:39.267154 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:39.267393 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:39.267394 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:39.267394 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:39.267548 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:39.267676 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:39.268068 (+ 392us) server_negotiation.cc:299] Negotiation successful
0619 02:43:39.268142 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 02:43:39.268602 26351 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:43:39.270831 26351 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2a456fd94c0a4d949653fce0dbd8a338: Generated new TSK 0
I20260619 02:43:39.271008 26351 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:43:39.273248 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 26272
W20260619 02:43:39.277462 26257 connection.cc:535] client connection to 127.0.0.1:46177 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46177): Connection reset by peer (error 104)
2026-06-19T02:43:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1326 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836772852275-6085-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:43:39 dist-test-slave-zpbg krb5kdc[26371](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:43:39 dist-test-slave-zpbg krb5kdc[26371](info): set up 2 sockets
Jun 19 02:43:39 dist-test-slave-zpbg krb5kdc[26371](info): commencing operation
krb5kdc: starting...
W20260619 02:43:40.310878 6085 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Jun 19 02:43:40 dist-test-slave-zpbg krb5kdc[26371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837020, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:43:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:43:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836772852275-6085-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836772852275-6085-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:43:40.433125 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41135
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836772852275-6085-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41135
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:43:40.514367 26384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:40.514597 26384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:40.514622 26384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:40.515758 26384 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:43:40.515797 26384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:40.515812 26384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:40.515826 26384 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:43:40.515838 26384 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:43:40.517235 26384 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836772852275-6085-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41135
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836772852275-6085-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26384
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:43:40.517488 26384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:40.517732 26384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:40.522831 26389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:40.522894 26390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:40.522847 26392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:40.522934 26384 server_base.cc:1034] running on GCE node
I20260619 02:43:40.523289 26384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:40.523594 26384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:40.524792 26384 hybrid_clock.cc:648] HybridClock initialized: now 1781837020524772 us; error 49 us; skew 500 ppm
Jun 19 02:43:40 dist-test-slave-zpbg krb5kdc[26371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837020, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:40.527496 26384 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:43:40.528060 26384 webserver.cc:460] Webserver started at http://127.0.0.1:41047/ using document root <none> and password file <none>
I20260619 02:43:40.528318 26384 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:40.528395 26384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:40.528568 26384 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:43:40.529844 26384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836772852275-6085-0/minicluster-data/master-0/data/instance:
uuid: "dc9fca6f77684a919f9b29997ead61e8"
format_stamp: "Formatted at 2026-06-19 02:43:40 on dist-test-slave-zpbg"
server_key: "6f92063664bdeb539e6f332f95ee466e"
server_key_iv: "cae1fcfc06347b11ff78c8329835578a"
server_key_version: "encryptionkey@0"
I20260619 02:43:40.530247 26384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836772852275-6085-0/minicluster-data/master-0/wal/instance:
uuid: "dc9fca6f77684a919f9b29997ead61e8"
format_stamp: "Formatted at 2026-06-19 02:43:40 on dist-test-slave-zpbg"
server_key: "6f92063664bdeb539e6f332f95ee466e"
server_key_iv: "cae1fcfc06347b11ff78c8329835578a"
server_key_version: "encryptionkey@0"
I20260619 02:43:40.531872 26384 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260619 02:43:40.532691 26399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:40.532871 26384 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:43:40.532989 26384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "dc9fca6f77684a919f9b29997ead61e8"
format_stamp: "Formatted at 2026-06-19 02:43:40 on dist-test-slave-zpbg"
server_key: "6f92063664bdeb539e6f332f95ee466e"
server_key_iv: "cae1fcfc06347b11ff78c8329835578a"
server_key_version: "encryptionkey@0"
I20260619 02:43:40.533077 26384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:40.548488 26384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:40.550686 26384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:40.550920 26384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:40.555239 26384 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41135
I20260619 02:43:40.555274 26451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41135 every 8 connection(s)
I20260619 02:43:40.555657 26384 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:43:40.556699 26452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:40.558075 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 26384
I20260619 02:43:40.558179 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836772852275-6085-0/minicluster-data/master-0/wal/instance
I20260619 02:43:40.558400 6085 external_mini_cluster.cc:1428] Setting key 45b82c1c4e97c179b4451905bfc46c44
I20260619 02:43:40.559363 26452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc9fca6f77684a919f9b29997ead61e8: Bootstrap starting.
I20260619 02:43:40.559937 26452 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc9fca6f77684a919f9b29997ead61e8: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:40.560240 26452 log.cc:824] T 00000000000000000000000000000000 P dc9fca6f77684a919f9b29997ead61e8: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:40.560969 26452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc9fca6f77684a919f9b29997ead61e8: No bootstrap required, opened a new log
I20260619 02:43:40.562716 26452 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc9fca6f77684a919f9b29997ead61e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc9fca6f77684a919f9b29997ead61e8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41135 } }
I20260619 02:43:40.562948 26452 raft_consensus.cc:374] T 00000000000000000000000000000000 P dc9fca6f77684a919f9b29997ead61e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:40.563002 26452 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc9fca6f77684a919f9b29997ead61e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc9fca6f77684a919f9b29997ead61e8, State: Initialized, Role: FOLLOWER
I20260619 02:43:40.563144 26452 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc9fca6f77684a919f9b29997ead61e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc9fca6f77684a919f9b29997ead61e8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41135 } }
I20260619 02:43:40.563231 26452 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc9fca6f77684a919f9b29997ead61e8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:43:40.563278 26452 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc9fca6f77684a919f9b29997ead61e8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:43:40.563431 26452 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc9fca6f77684a919f9b29997ead61e8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:40.564136 26452 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc9fca6f77684a919f9b29997ead61e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc9fca6f77684a919f9b29997ead61e8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41135 } }
I20260619 02:43:40.564289 26455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:40.558946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60839 (local address 127.0.0.1:41135)
0619 02:43:40.559140 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:40.559149 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:40.559185 (+ 36us) server_negotiation.cc:407] Connection header received
0619 02:43:40.559635 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:40.559642 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:40.559692 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:40.559850 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:40.560075 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:40.561223 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:40.562025 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:40.562451 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:40.562588 (+ 137us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:40.563524 (+ 936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:40.563529 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:40.563529 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:43:40.563723 (+ 194us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:40.563755 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:43:40.563766 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:40.563842 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:40.563971 (+ 129us) server_negotiation.cc:299] Negotiation successful
0619 02:43:40.564022 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":82,"threads_started":1}
I20260619 02:43:40.564256 26452 leader_election.cc:302] T 00000000000000000000000000000000 P dc9fca6f77684a919f9b29997ead61e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc9fca6f77684a919f9b29997ead61e8; no voters:
I20260619 02:43:40.564507 26452 leader_election.cc:288] T 00000000000000000000000000000000 P dc9fca6f77684a919f9b29997ead61e8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:43:40.564543 26457 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc9fca6f77684a919f9b29997ead61e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:43:40.564680 26457 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc9fca6f77684a919f9b29997ead61e8 [term 1 LEADER]: Becoming Leader. State: Replica: dc9fca6f77684a919f9b29997ead61e8, State: Running, Role: LEADER
I20260619 02:43:40.564786 26452 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc9fca6f77684a919f9b29997ead61e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:43:40.564827 26457 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc9fca6f77684a919f9b29997ead61e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc9fca6f77684a919f9b29997ead61e8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41135 } }
I20260619 02:43:40.565248 6085 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:43:40.565339 26458 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc9fca6f77684a919f9b29997ead61e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dc9fca6f77684a919f9b29997ead61e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc9fca6f77684a919f9b29997ead61e8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41135 } } }
I20260619 02:43:40.565412 26458 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc9fca6f77684a919f9b29997ead61e8 [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:40.565472 26457 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc9fca6f77684a919f9b29997ead61e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc9fca6f77684a919f9b29997ead61e8. Latest consensus state: current_term: 1 leader_uuid: "dc9fca6f77684a919f9b29997ead61e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc9fca6f77684a919f9b29997ead61e8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41135 } } }
I20260619 02:43:40.565840 26464 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:43:40.566356 26457 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc9fca6f77684a919f9b29997ead61e8 [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:40.566970 26464 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:43:40.568549 26464 catalog_manager.cc:1347] Generated new cluster ID: c18d37bccce549ee888f71ce5ecf01d1
I20260619 02:43:40.568600 26464 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:43:40.571403 26455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:40.567899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57887 (local address 127.0.0.1:41135)
0619 02:43:40.567933 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:40.567934 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:40.567943 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:43:40.568024 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:40.568024 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:40.568052 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:40.568117 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:40.568272 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:40.569004 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:40.570037 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:40.570332 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:40.570392 (+ 60us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:40.571134 (+ 742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:40.571135 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:40.571136 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:43:40.571140 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:40.571155 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:43:40.571160 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:40.571213 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:40.571323 (+ 110us) server_negotiation.cc:299] Negotiation successful
0619 02:43:40.571333 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 02:43:40.590914 26464 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:43:40.591539 26464 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:43:40.594565 26464 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dc9fca6f77684a919f9b29997ead61e8: Generated new TSK 0
I20260619 02:43:40.594817 26464 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:43:40.601310 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 26384
W20260619 02:43:40.605822 26369 connection.cc:535] client connection to 127.0.0.1:41135 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41135): Connection reset by peer (error 104)
2026-06-19T02:43:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1328 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836772852275-6085-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:43:40 dist-test-slave-zpbg krb5kdc[26483](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:43:40 dist-test-slave-zpbg krb5kdc[26483](info): set up 2 sockets
Jun 19 02:43:40 dist-test-slave-zpbg krb5kdc[26483](info): commencing operation
krb5kdc: starting...
W20260619 02:43:41.654847 6085 mini_kdc.cc:121] Time spent starting KDC: real 1.035s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Jun 19 02:43:41 dist-test-slave-zpbg krb5kdc[26483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837021, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:43:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:43:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836772852275-6085-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836772852275-6085-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:43:41.773939 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39379
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836772852275-6085-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39379
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:43:41.853566 26496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:41.853832 26496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:41.853874 26496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:41.855046 26496 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:43:41.855121 26496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:41.855146 26496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:41.855180 26496 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:43:41.855206 26496 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:43:41.856680 26496 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836772852275-6085-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39379
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836772852275-6085-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39379
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26496
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:43:41.856962 26496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:41.857187 26496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:41.862105 26504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:41.862111 26501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:41.862100 26502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:41.862442 26496 server_base.cc:1034] running on GCE node
I20260619 02:43:41.862676 26496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:41.862987 26496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:41.864154 26496 hybrid_clock.cc:648] HybridClock initialized: now 1781837021864132 us; error 35 us; skew 500 ppm
Jun 19 02:43:41 dist-test-slave-zpbg krb5kdc[26483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837021, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:41.866525 26496 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:43:41.867018 26496 webserver.cc:460] Webserver started at http://127.0.0.1:39385/ using document root <none> and password file <none>
I20260619 02:43:41.867255 26496 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:41.867324 26496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:41.867467 26496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:43:41.868512 26496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836772852275-6085-0/minicluster-data/master-0/data/instance:
uuid: "295a59a7d7d2470bad8852549097d7bb"
format_stamp: "Formatted at 2026-06-19 02:43:41 on dist-test-slave-zpbg"
server_key: "3b2dc56250a385c60031fa45f57bd113"
server_key_iv: "64778760fa5a42d5d5b78ee0fc756404"
server_key_version: "encryptionkey@0"
I20260619 02:43:41.868875 26496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836772852275-6085-0/minicluster-data/master-0/wal/instance:
uuid: "295a59a7d7d2470bad8852549097d7bb"
format_stamp: "Formatted at 2026-06-19 02:43:41 on dist-test-slave-zpbg"
server_key: "3b2dc56250a385c60031fa45f57bd113"
server_key_iv: "64778760fa5a42d5d5b78ee0fc756404"
server_key_version: "encryptionkey@0"
I20260619 02:43:41.870446 26496 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:43:41.871379 26511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:41.871618 26496 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 02:43:41.871735 26496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "295a59a7d7d2470bad8852549097d7bb"
format_stamp: "Formatted at 2026-06-19 02:43:41 on dist-test-slave-zpbg"
server_key: "3b2dc56250a385c60031fa45f57bd113"
server_key_iv: "64778760fa5a42d5d5b78ee0fc756404"
server_key_version: "encryptionkey@0"
I20260619 02:43:41.871829 26496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:41.875237 26496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:41.877271 26496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:41.877432 26496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:41.881512 26496 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39379
I20260619 02:43:41.881551 26563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39379 every 8 connection(s)
I20260619 02:43:41.881889 26496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:43:41.882908 26564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:41.885083 26564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 295a59a7d7d2470bad8852549097d7bb: Bootstrap starting.
I20260619 02:43:41.885643 26564 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 295a59a7d7d2470bad8852549097d7bb: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:41.885919 26564 log.cc:824] T 00000000000000000000000000000000 P 295a59a7d7d2470bad8852549097d7bb: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:41.886581 26564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 295a59a7d7d2470bad8852549097d7bb: No bootstrap required, opened a new log
I20260619 02:43:41.888221 26564 raft_consensus.cc:348] T 00000000000000000000000000000000 P 295a59a7d7d2470bad8852549097d7bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "295a59a7d7d2470bad8852549097d7bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39379 } }
I20260619 02:43:41.888401 26564 raft_consensus.cc:374] T 00000000000000000000000000000000 P 295a59a7d7d2470bad8852549097d7bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:41.888440 26564 raft_consensus.cc:729] T 00000000000000000000000000000000 P 295a59a7d7d2470bad8852549097d7bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 295a59a7d7d2470bad8852549097d7bb, State: Initialized, Role: FOLLOWER
I20260619 02:43:41.888517 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 26496
I20260619 02:43:41.888645 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836772852275-6085-0/minicluster-data/master-0/wal/instance
I20260619 02:43:41.888592 26564 consensus_queue.cc:260] T 00000000000000000000000000000000 P 295a59a7d7d2470bad8852549097d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "295a59a7d7d2470bad8852549097d7bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39379 } }
I20260619 02:43:41.888685 26564 raft_consensus.cc:388] T 00000000000000000000000000000000 P 295a59a7d7d2470bad8852549097d7bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:43:41.888721 26564 raft_consensus.cc:482] T 00000000000000000000000000000000 P 295a59a7d7d2470bad8852549097d7bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:43:41.888777 26564 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 295a59a7d7d2470bad8852549097d7bb [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:41.888854 6085 external_mini_cluster.cc:1428] Setting key 1107ef487a89afec2a1bd06fdf51fb39
I20260619 02:43:41.889396 26564 raft_consensus.cc:504] T 00000000000000000000000000000000 P 295a59a7d7d2470bad8852549097d7bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "295a59a7d7d2470bad8852549097d7bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39379 } }
I20260619 02:43:41.889561 26564 leader_election.cc:302] T 00000000000000000000000000000000 P 295a59a7d7d2470bad8852549097d7bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 295a59a7d7d2470bad8852549097d7bb; no voters:
I20260619 02:43:41.889732 26564 leader_election.cc:288] T 00000000000000000000000000000000 P 295a59a7d7d2470bad8852549097d7bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:43:41.889760 26569 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 295a59a7d7d2470bad8852549097d7bb [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:43:41.889959 26569 raft_consensus.cc:686] T 00000000000000000000000000000000 P 295a59a7d7d2470bad8852549097d7bb [term 1 LEADER]: Becoming Leader. State: Replica: 295a59a7d7d2470bad8852549097d7bb, State: Running, Role: LEADER
I20260619 02:43:41.889981 26564 sys_catalog.cc:564] T 00000000000000000000000000000000 P 295a59a7d7d2470bad8852549097d7bb [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:43:41.890057 26569 consensus_queue.cc:237] T 00000000000000000000000000000000 P 295a59a7d7d2470bad8852549097d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "295a59a7d7d2470bad8852549097d7bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39379 } }
I20260619 02:43:41.890486 26569 sys_catalog.cc:455] T 00000000000000000000000000000000 P 295a59a7d7d2470bad8852549097d7bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 295a59a7d7d2470bad8852549097d7bb. Latest consensus state: current_term: 1 leader_uuid: "295a59a7d7d2470bad8852549097d7bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "295a59a7d7d2470bad8852549097d7bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39379 } } }
I20260619 02:43:41.890496 26570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 295a59a7d7d2470bad8852549097d7bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "295a59a7d7d2470bad8852549097d7bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "295a59a7d7d2470bad8852549097d7bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39379 } } }
I20260619 02:43:41.890678 26570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 295a59a7d7d2470bad8852549097d7bb [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:41.890645 26569 sys_catalog.cc:458] T 00000000000000000000000000000000 P 295a59a7d7d2470bad8852549097d7bb [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:41.891095 26575 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:43:41.891855 26568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:41.889339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51517 (local address 127.0.0.1:39379)
0619 02:43:41.889521 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:41.889531 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:41.889562 (+ 31us) server_negotiation.cc:407] Connection header received
0619 02:43:41.889698 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:41.889706 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:41.889739 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:41.890152 (+ 413us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:41.891364 (+ 1212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:41.891367 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:41.891368 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:43:41.891373 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:41.891417 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:43:41.891432 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:41.891493 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:41.891623 (+ 130us) server_negotiation.cc:299] Negotiation successful
0619 02:43:41.891676 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":91,"threads_started":1}
I20260619 02:43:41.891877 26575 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:43:41.892788 6085 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:43:41.893433 26575 catalog_manager.cc:1347] Generated new cluster ID: 28f89d00a7334245958ffc6e0e00a841
I20260619 02:43:41.893491 26575 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:43:41.903560 26568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:41.894227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44397 (local address 127.0.0.1:39379)
0619 02:43:41.894269 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:41.894271 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:41.894459 (+ 188us) server_negotiation.cc:407] Connection header received
0619 02:43:41.894556 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:41.894557 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:41.894563 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:41.894626 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:41.903271 (+ 8645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:41.903273 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:41.903273 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:43:41.903280 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:41.903297 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:43:41.903302 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:41.903361 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:41.903436 (+ 75us) server_negotiation.cc:299] Negotiation successful
0619 02:43:41.903451 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 02:43:41.906513 26575 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:43:41.906697 26575 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:43:41.910064 26575 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 295a59a7d7d2470bad8852549097d7bb: Generated new TSK 0
I20260619 02:43:41.910246 26575 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:43:41.912488 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 26496
2026-06-19T02:43:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1310 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836772852275-6085-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:43:41 dist-test-slave-zpbg krb5kdc[26595](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:43:41 dist-test-slave-zpbg krb5kdc[26595](info): set up 2 sockets
Jun 19 02:43:41 dist-test-slave-zpbg krb5kdc[26595](info): commencing operation
krb5kdc: starting...
W20260619 02:43:43.034909 6085 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Jun 19 02:43:43 dist-test-slave-zpbg krb5kdc[26595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:43:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:43:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836772852275-6085-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836772852275-6085-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:43:43.158417 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39521
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836772852275-6085-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39521
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:43:43.237723 26608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:43.237924 26608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:43.237948 26608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:43.239143 26608 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:43:43.239189 26608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:43.239203 26608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:43.239216 26608 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:43:43.239228 26608 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:43:43.240705 26608 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836772852275-6085-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39521
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836772852275-6085-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26608
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:43:43.240949 26608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:43.241185 26608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:43.245961 26613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:43.245961 26614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:43.246002 26616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:43.246202 26608 server_base.cc:1034] running on GCE node
I20260619 02:43:43.246495 26608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:43.246820 26608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:43.247973 26608 hybrid_clock.cc:648] HybridClock initialized: now 1781837023247953 us; error 37 us; skew 500 ppm
Jun 19 02:43:43 dist-test-slave-zpbg krb5kdc[26595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837023, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:43.256546 26608 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:43:43.257082 26608 webserver.cc:460] Webserver started at http://127.0.0.1:41789/ using document root <none> and password file <none>
I20260619 02:43:43.257333 26608 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:43.257402 26608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:43.257544 26608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:43:43.258642 26608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836772852275-6085-0/minicluster-data/master-0/data/instance:
uuid: "304975e9c9ae4e70b0f10150ee00da67"
format_stamp: "Formatted at 2026-06-19 02:43:43 on dist-test-slave-zpbg"
server_key: "6061128b219616d3dc9d297c154d6070"
server_key_iv: "35e5c60843fa983f6d1310137e54f476"
server_key_version: "encryptionkey@0"
I20260619 02:43:43.259042 26608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836772852275-6085-0/minicluster-data/master-0/wal/instance:
uuid: "304975e9c9ae4e70b0f10150ee00da67"
format_stamp: "Formatted at 2026-06-19 02:43:43 on dist-test-slave-zpbg"
server_key: "6061128b219616d3dc9d297c154d6070"
server_key_iv: "35e5c60843fa983f6d1310137e54f476"
server_key_version: "encryptionkey@0"
I20260619 02:43:43.260354 26608 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:43:43.261148 26623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:43.261345 26608 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:43:43.261438 26608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "304975e9c9ae4e70b0f10150ee00da67"
format_stamp: "Formatted at 2026-06-19 02:43:43 on dist-test-slave-zpbg"
server_key: "6061128b219616d3dc9d297c154d6070"
server_key_iv: "35e5c60843fa983f6d1310137e54f476"
server_key_version: "encryptionkey@0"
I20260619 02:43:43.261518 26608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:43.286412 26608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:43.288570 26608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:43.288748 26608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:43.292917 26608 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39521
I20260619 02:43:43.292981 26675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39521 every 8 connection(s)
I20260619 02:43:43.293355 26608 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:43:43.294356 26676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:43.296602 26676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 304975e9c9ae4e70b0f10150ee00da67: Bootstrap starting.
I20260619 02:43:43.297128 26676 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 304975e9c9ae4e70b0f10150ee00da67: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:43.297354 26676 log.cc:824] T 00000000000000000000000000000000 P 304975e9c9ae4e70b0f10150ee00da67: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:43.298095 26676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 304975e9c9ae4e70b0f10150ee00da67: No bootstrap required, opened a new log
I20260619 02:43:43.299849 26676 raft_consensus.cc:348] T 00000000000000000000000000000000 P 304975e9c9ae4e70b0f10150ee00da67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "304975e9c9ae4e70b0f10150ee00da67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39521 } }
I20260619 02:43:43.300001 26676 raft_consensus.cc:374] T 00000000000000000000000000000000 P 304975e9c9ae4e70b0f10150ee00da67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:43.300028 26676 raft_consensus.cc:729] T 00000000000000000000000000000000 P 304975e9c9ae4e70b0f10150ee00da67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 304975e9c9ae4e70b0f10150ee00da67, State: Initialized, Role: FOLLOWER
I20260619 02:43:43.300148 26676 consensus_queue.cc:260] T 00000000000000000000000000000000 P 304975e9c9ae4e70b0f10150ee00da67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "304975e9c9ae4e70b0f10150ee00da67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39521 } }
I20260619 02:43:43.300203 26676 raft_consensus.cc:388] T 00000000000000000000000000000000 P 304975e9c9ae4e70b0f10150ee00da67 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:43:43.300227 26676 raft_consensus.cc:482] T 00000000000000000000000000000000 P 304975e9c9ae4e70b0f10150ee00da67 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:43:43.300262 26676 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 304975e9c9ae4e70b0f10150ee00da67 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:43.300779 26676 raft_consensus.cc:504] T 00000000000000000000000000000000 P 304975e9c9ae4e70b0f10150ee00da67 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "304975e9c9ae4e70b0f10150ee00da67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39521 } }
I20260619 02:43:43.300894 26676 leader_election.cc:302] T 00000000000000000000000000000000 P 304975e9c9ae4e70b0f10150ee00da67 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 304975e9c9ae4e70b0f10150ee00da67; no voters:
I20260619 02:43:43.301040 26676 leader_election.cc:288] T 00000000000000000000000000000000 P 304975e9c9ae4e70b0f10150ee00da67 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:43:43.301126 26679 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 304975e9c9ae4e70b0f10150ee00da67 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:43:43.301330 26676 sys_catalog.cc:564] T 00000000000000000000000000000000 P 304975e9c9ae4e70b0f10150ee00da67 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:43:43.301333 26679 raft_consensus.cc:686] T 00000000000000000000000000000000 P 304975e9c9ae4e70b0f10150ee00da67 [term 1 LEADER]: Becoming Leader. State: Replica: 304975e9c9ae4e70b0f10150ee00da67, State: Running, Role: LEADER
I20260619 02:43:43.301453 26679 consensus_queue.cc:237] T 00000000000000000000000000000000 P 304975e9c9ae4e70b0f10150ee00da67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "304975e9c9ae4e70b0f10150ee00da67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39521 } }
I20260619 02:43:43.301890 26681 sys_catalog.cc:455] T 00000000000000000000000000000000 P 304975e9c9ae4e70b0f10150ee00da67 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 304975e9c9ae4e70b0f10150ee00da67. Latest consensus state: current_term: 1 leader_uuid: "304975e9c9ae4e70b0f10150ee00da67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "304975e9c9ae4e70b0f10150ee00da67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39521 } } }
I20260619 02:43:43.302084 26681 sys_catalog.cc:458] T 00000000000000000000000000000000 P 304975e9c9ae4e70b0f10150ee00da67 [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:43.301918 26680 sys_catalog.cc:455] T 00000000000000000000000000000000 P 304975e9c9ae4e70b0f10150ee00da67 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "304975e9c9ae4e70b0f10150ee00da67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "304975e9c9ae4e70b0f10150ee00da67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39521 } } }
I20260619 02:43:43.302276 26680 sys_catalog.cc:458] T 00000000000000000000000000000000 P 304975e9c9ae4e70b0f10150ee00da67 [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:43.302541 26683 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:43:43.303155 26683 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:43:43.303362 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 26608
I20260619 02:43:43.303462 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836772852275-6085-0/minicluster-data/master-0/wal/instance
I20260619 02:43:43.303655 6085 external_mini_cluster.cc:1428] Setting key 4a4b38a10bbc3cf9f6b703563f674a5a
I20260619 02:43:43.309379 26683 catalog_manager.cc:1347] Generated new cluster ID: a6276a0d27d14b3b99f91104863284e4
I20260619 02:43:43.309440 26683 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 02:43:43 dist-test-slave-zpbg krb5kdc[26595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 02:43:43.315795 26699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:43.304479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55973 (local address 127.0.0.1:39521)
0619 02:43:43.304634 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:43.304641 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:43.304694 (+ 53us) server_negotiation.cc:407] Connection header received
0619 02:43:43.304838 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:43.304843 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:43.304892 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:43.305006 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:43.305780 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:43.306700 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:43.307860 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:43.308293 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:43.308390 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:43.312432 (+ 4042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:43.312436 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:43.312437 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:43.312450 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:43.314177 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:43.314546 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:43.314549 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:43.314550 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:43.314671 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:43.314925 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:43.314926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:43.314926 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:43.315170 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:43.315349 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:43.315430 (+ 81us) server_negotiation.cc:299] Negotiation successful
0619 02:43:43.315574 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":80,"threads_started":1}
I20260619 02:43:43.316752 6085 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:43:43.324412 26699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:43.318069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45825 (local address 127.0.0.1:39521)
0619 02:43:43.318104 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:43.318106 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:43.318291 (+ 185us) server_negotiation.cc:407] Connection header received
0619 02:43:43.318336 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:43.318336 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:43.318362 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:43.318405 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:43.318993 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:43.319512 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:43.320242 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:43.320488 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:43.320569 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:43.321761 (+ 1192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:43.321763 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:43.321763 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:43.321770 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:43.323243 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:43.323619 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:43.323620 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:43.323621 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:43.323683 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:43.323893 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:43.323894 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:43.323894 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:43.324010 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:43.324113 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:43.324229 (+ 116us) server_negotiation.cc:299] Negotiation successful
0619 02:43:43.324306 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 02:43:43.340672 26683 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:43:43.341394 26683 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:43:43.350081 26683 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 304975e9c9ae4e70b0f10150ee00da67: Generated new TSK 0
I20260619 02:43:43.350288 26683 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:43:43.354882 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 26608
2026-06-19T02:43:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1442 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836772852275-6085-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:43:43 dist-test-slave-zpbg krb5kdc[26707](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:43:43 dist-test-slave-zpbg krb5kdc[26707](info): set up 2 sockets
Jun 19 02:43:43 dist-test-slave-zpbg krb5kdc[26707](info): commencing operation
krb5kdc: starting...
W20260619 02:43:44.390829 6085 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Jun 19 02:43:44 dist-test-slave-zpbg krb5kdc[26707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:43:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:43:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836772852275-6085-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836772852275-6085-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:43:44.515107 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44201
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836772852275-6085-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44201
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:43:44.597801 26720 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:44.598059 26720 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:44.598101 26720 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:44.599299 26720 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:43:44.599375 26720 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:44.599400 26720 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:44.599428 26720 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:43:44.599457 26720 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:43:44.600950 26720 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836772852275-6085-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44201
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836772852275-6085-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26720
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:43:44.601228 26720 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:44.601461 26720 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:44.606479 26726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:44.606487 26725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:44.606496 26728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:44.606583 26720 server_base.cc:1034] running on GCE node
I20260619 02:43:44.606971 26720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:44.607219 26720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:44.608368 26720 hybrid_clock.cc:648] HybridClock initialized: now 1781837024608353 us; error 30 us; skew 500 ppm
Jun 19 02:43:44 dist-test-slave-zpbg krb5kdc[26707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837024, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:44.610940 26720 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:43:44.611389 26720 webserver.cc:460] Webserver started at http://127.0.0.1:44217/ using document root <none> and password file <none>
I20260619 02:43:44.611644 26720 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:44.611716 26720 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:44.611879 26720 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:43:44.612958 26720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836772852275-6085-0/minicluster-data/master-0/data/instance:
uuid: "1867f5ab38e8426bbd269c2066783ca1"
format_stamp: "Formatted at 2026-06-19 02:43:44 on dist-test-slave-zpbg"
server_key: "072dfccf3663783ce32010a1febc2012"
server_key_iv: "d60179d9a3539243eca55040c83cebf4"
server_key_version: "encryptionkey@0"
I20260619 02:43:44.613343 26720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836772852275-6085-0/minicluster-data/master-0/wal/instance:
uuid: "1867f5ab38e8426bbd269c2066783ca1"
format_stamp: "Formatted at 2026-06-19 02:43:44 on dist-test-slave-zpbg"
server_key: "072dfccf3663783ce32010a1febc2012"
server_key_iv: "d60179d9a3539243eca55040c83cebf4"
server_key_version: "encryptionkey@0"
I20260619 02:43:44.614722 26720 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:43:44.615571 26735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:44.615790 26720 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:43:44.615899 26720 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "1867f5ab38e8426bbd269c2066783ca1"
format_stamp: "Formatted at 2026-06-19 02:43:44 on dist-test-slave-zpbg"
server_key: "072dfccf3663783ce32010a1febc2012"
server_key_iv: "d60179d9a3539243eca55040c83cebf4"
server_key_version: "encryptionkey@0"
I20260619 02:43:44.615975 26720 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:44.626293 26720 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:44.628609 26720 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:44.628830 26720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:44.633445 26720 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44201
I20260619 02:43:44.633481 26787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44201 every 8 connection(s)
I20260619 02:43:44.633846 26720 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:43:44.634886 26788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:44.637288 26788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1867f5ab38e8426bbd269c2066783ca1: Bootstrap starting.
I20260619 02:43:44.637888 26788 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1867f5ab38e8426bbd269c2066783ca1: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:44.638188 26788 log.cc:824] T 00000000000000000000000000000000 P 1867f5ab38e8426bbd269c2066783ca1: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:44.639206 26788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1867f5ab38e8426bbd269c2066783ca1: No bootstrap required, opened a new log
I20260619 02:43:44.640426 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 26720
I20260619 02:43:44.640602 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836772852275-6085-0/minicluster-data/master-0/wal/instance
I20260619 02:43:44.640851 6085 external_mini_cluster.cc:1428] Setting key 2d07d6e51c495216c90a3a8bd4960a38
I20260619 02:43:44.641641 26788 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1867f5ab38e8426bbd269c2066783ca1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1867f5ab38e8426bbd269c2066783ca1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44201 } }
I20260619 02:43:44.641829 26788 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1867f5ab38e8426bbd269c2066783ca1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:44.641877 26788 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1867f5ab38e8426bbd269c2066783ca1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1867f5ab38e8426bbd269c2066783ca1, State: Initialized, Role: FOLLOWER
I20260619 02:43:44.642038 26788 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1867f5ab38e8426bbd269c2066783ca1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1867f5ab38e8426bbd269c2066783ca1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44201 } }
I20260619 02:43:44.642123 26788 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1867f5ab38e8426bbd269c2066783ca1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:43:44.642171 26788 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1867f5ab38e8426bbd269c2066783ca1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:43:44.642225 26788 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1867f5ab38e8426bbd269c2066783ca1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:44.643040 26788 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1867f5ab38e8426bbd269c2066783ca1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1867f5ab38e8426bbd269c2066783ca1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44201 } }
I20260619 02:43:44.643165 26788 leader_election.cc:302] T 00000000000000000000000000000000 P 1867f5ab38e8426bbd269c2066783ca1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1867f5ab38e8426bbd269c2066783ca1; no voters:
I20260619 02:43:44.643328 26788 leader_election.cc:288] T 00000000000000000000000000000000 P 1867f5ab38e8426bbd269c2066783ca1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:43:44.643404 26793 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1867f5ab38e8426bbd269c2066783ca1 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:43:44.643594 26788 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1867f5ab38e8426bbd269c2066783ca1 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:43:44.643597 26793 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1867f5ab38e8426bbd269c2066783ca1 [term 1 LEADER]: Becoming Leader. State: Replica: 1867f5ab38e8426bbd269c2066783ca1, State: Running, Role: LEADER
I20260619 02:43:44.643873 26793 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1867f5ab38e8426bbd269c2066783ca1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1867f5ab38e8426bbd269c2066783ca1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44201 } }
I20260619 02:43:44.644290 26795 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1867f5ab38e8426bbd269c2066783ca1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1867f5ab38e8426bbd269c2066783ca1. Latest consensus state: current_term: 1 leader_uuid: "1867f5ab38e8426bbd269c2066783ca1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1867f5ab38e8426bbd269c2066783ca1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44201 } } }
I20260619 02:43:44.644368 26795 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1867f5ab38e8426bbd269c2066783ca1 [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:44.644703 26794 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1867f5ab38e8426bbd269c2066783ca1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1867f5ab38e8426bbd269c2066783ca1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1867f5ab38e8426bbd269c2066783ca1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44201 } } }
I20260619 02:43:44.644773 26794 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1867f5ab38e8426bbd269c2066783ca1 [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:44.645149 26800 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:43:44.645731 26800 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:43:44.648628 26800 catalog_manager.cc:1347] Generated new cluster ID: a9b2972521d24dacac2af2f821dd8b33
I20260619 02:43:44.648699 26800 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:43:44.649324 26792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:44.641454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45075 (local address 127.0.0.1:44201)
0619 02:43:44.641655 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:44.641664 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:44.641690 (+ 26us) server_negotiation.cc:407] Connection header received
0619 02:43:44.641790 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:44.641796 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:44.641841 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:44.642036 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:44.642737 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:44.644268 (+ 1531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:44.646328 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:44.647263 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:44.647426 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:44.648523 (+ 1097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:44.648526 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:44.648527 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:43:44.648780 (+ 253us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:44.648819 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:43:44.648835 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:44.648910 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:44.649031 (+ 121us) server_negotiation.cc:299] Negotiation successful
0619 02:43:44.649096 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":110,"threads_started":1}
I20260619 02:43:44.650333 6085 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:43:44.655750 26792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:44.651764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52235 (local address 127.0.0.1:44201)
0619 02:43:44.651813 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:44.651815 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:44.651860 (+ 45us) server_negotiation.cc:407] Connection header received
0619 02:43:44.651932 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:44.651933 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:44.651944 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:44.651996 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:44.652165 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:44.653132 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:44.654359 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:44.654649 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:44.654710 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:44.655483 (+ 773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:44.655484 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:44.655484 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:43:44.655497 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:44.655512 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:43:44.655516 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:44.655568 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:44.655648 (+ 80us) server_negotiation.cc:299] Negotiation successful
0619 02:43:44.655674 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 02:43:44.661123 26800 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:43:44.661674 26800 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:43:44.666271 26800 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1867f5ab38e8426bbd269c2066783ca1: Generated new TSK 0
I20260619 02:43:44.666455 26800 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:43:44.673207 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 26720
2026-06-19T02:43:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1317 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836772852275-6085-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:43:44 dist-test-slave-zpbg krb5kdc[26819](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:43:44 dist-test-slave-zpbg krb5kdc[26819](info): set up 2 sockets
Jun 19 02:43:44 dist-test-slave-zpbg krb5kdc[26819](info): commencing operation
krb5kdc: starting...
W20260619 02:43:45.723018 6085 mini_kdc.cc:121] Time spent starting KDC: real 1.032s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Jun 19 02:43:45 dist-test-slave-zpbg krb5kdc[26819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:43:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:43:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836772852275-6085-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836772852275-6085-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:43:45.846343 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33875
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836772852275-6085-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33875
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:43:45.926083 26832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:45.926311 26832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:45.926333 26832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:45.927675 26832 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:43:45.927721 26832 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:45.927735 26832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:45.927749 26832 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:43:45.927767 26832 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:43:45.929298 26832 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836772852275-6085-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33875
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836772852275-6085-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26832
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:43:45.929587 26832 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:45.929829 26832 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:45.934856 26840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:45.935027 26832 server_base.cc:1034] running on GCE node
W20260619 02:43:45.934860 26838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:45.934885 26837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:45.935374 26832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:45.935705 26832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:45.936901 26832 hybrid_clock.cc:648] HybridClock initialized: now 1781837025936878 us; error 36 us; skew 500 ppm
Jun 19 02:43:45 dist-test-slave-zpbg krb5kdc[26819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837025, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:45.939755 26832 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:43:45.940258 26832 webserver.cc:460] Webserver started at http://127.0.0.1:42875/ using document root <none> and password file <none>
I20260619 02:43:45.940505 26832 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:45.940574 26832 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:45.940724 26832 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:43:45.941828 26832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836772852275-6085-0/minicluster-data/master-0/data/instance:
uuid: "3beb4b845fcc4fe78ba6d2fb7d5f6c9b"
format_stamp: "Formatted at 2026-06-19 02:43:45 on dist-test-slave-zpbg"
server_key: "147d6831733ba7883d75798437b9fe1a"
server_key_iv: "a9069e246fd4e0638732bcd404941775"
server_key_version: "encryptionkey@0"
I20260619 02:43:45.942207 26832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836772852275-6085-0/minicluster-data/master-0/wal/instance:
uuid: "3beb4b845fcc4fe78ba6d2fb7d5f6c9b"
format_stamp: "Formatted at 2026-06-19 02:43:45 on dist-test-slave-zpbg"
server_key: "147d6831733ba7883d75798437b9fe1a"
server_key_iv: "a9069e246fd4e0638732bcd404941775"
server_key_version: "encryptionkey@0"
I20260619 02:43:45.943775 26832 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260619 02:43:45.944541 26847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:45.944732 26832 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:43:45.944839 26832 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "3beb4b845fcc4fe78ba6d2fb7d5f6c9b"
format_stamp: "Formatted at 2026-06-19 02:43:45 on dist-test-slave-zpbg"
server_key: "147d6831733ba7883d75798437b9fe1a"
server_key_iv: "a9069e246fd4e0638732bcd404941775"
server_key_version: "encryptionkey@0"
I20260619 02:43:45.944913 26832 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:45.948305 26832 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:45.950636 26832 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:45.950847 26832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:45.955173 26832 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33875
I20260619 02:43:45.955236 26899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33875 every 8 connection(s)
I20260619 02:43:45.955590 26832 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:43:45.956532 26900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:45.958815 26900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3beb4b845fcc4fe78ba6d2fb7d5f6c9b: Bootstrap starting.
I20260619 02:43:45.959396 26900 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3beb4b845fcc4fe78ba6d2fb7d5f6c9b: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:45.959685 26900 log.cc:824] T 00000000000000000000000000000000 P 3beb4b845fcc4fe78ba6d2fb7d5f6c9b: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:45.960357 26900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3beb4b845fcc4fe78ba6d2fb7d5f6c9b: No bootstrap required, opened a new log
I20260619 02:43:45.960835 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 26832
I20260619 02:43:45.960983 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836772852275-6085-0/minicluster-data/master-0/wal/instance
I20260619 02:43:45.961194 6085 external_mini_cluster.cc:1428] Setting key 3e57421b59118da2175f53ae1d93d430
I20260619 02:43:45.962316 26900 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3beb4b845fcc4fe78ba6d2fb7d5f6c9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3beb4b845fcc4fe78ba6d2fb7d5f6c9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33875 } }
I20260619 02:43:45.962493 26900 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3beb4b845fcc4fe78ba6d2fb7d5f6c9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:45.962536 26900 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3beb4b845fcc4fe78ba6d2fb7d5f6c9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3beb4b845fcc4fe78ba6d2fb7d5f6c9b, State: Initialized, Role: FOLLOWER
I20260619 02:43:45.962730 26900 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3beb4b845fcc4fe78ba6d2fb7d5f6c9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3beb4b845fcc4fe78ba6d2fb7d5f6c9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33875 } }
I20260619 02:43:45.962873 26900 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3beb4b845fcc4fe78ba6d2fb7d5f6c9b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:43:45.962975 26900 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3beb4b845fcc4fe78ba6d2fb7d5f6c9b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:43:45.963056 26900 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3beb4b845fcc4fe78ba6d2fb7d5f6c9b [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:45.963760 26904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:45.961808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36905 (local address 127.0.0.1:33875)
0619 02:43:45.961974 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:45.961983 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:45.962014 (+ 31us) server_negotiation.cc:407] Connection header received
0619 02:43:45.962297 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:45.962303 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:45.962328 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:45.962441 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:45.963197 (+ 756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:45.963201 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:45.963202 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:43:45.963206 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:45.963250 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:43:45.963265 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:45.963337 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:45.963461 (+ 124us) server_negotiation.cc:299] Negotiation successful
0619 02:43:45.963503 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":76,"threads_started":1}
I20260619 02:43:45.964250 26900 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3beb4b845fcc4fe78ba6d2fb7d5f6c9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3beb4b845fcc4fe78ba6d2fb7d5f6c9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33875 } }
I20260619 02:43:45.964376 26900 leader_election.cc:302] T 00000000000000000000000000000000 P 3beb4b845fcc4fe78ba6d2fb7d5f6c9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3beb4b845fcc4fe78ba6d2fb7d5f6c9b; no voters:
I20260619 02:43:45.964561 26900 leader_election.cc:288] T 00000000000000000000000000000000 P 3beb4b845fcc4fe78ba6d2fb7d5f6c9b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:43:45.964614 26905 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3beb4b845fcc4fe78ba6d2fb7d5f6c9b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:43:45.964766 6085 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:43:45.964803 26905 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3beb4b845fcc4fe78ba6d2fb7d5f6c9b [term 1 LEADER]: Becoming Leader. State: Replica: 3beb4b845fcc4fe78ba6d2fb7d5f6c9b, State: Running, Role: LEADER
I20260619 02:43:45.964816 26900 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3beb4b845fcc4fe78ba6d2fb7d5f6c9b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:43:45.964893 26905 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3beb4b845fcc4fe78ba6d2fb7d5f6c9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3beb4b845fcc4fe78ba6d2fb7d5f6c9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33875 } }
I20260619 02:43:45.965206 26905 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3beb4b845fcc4fe78ba6d2fb7d5f6c9b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3beb4b845fcc4fe78ba6d2fb7d5f6c9b. Latest consensus state: current_term: 1 leader_uuid: "3beb4b845fcc4fe78ba6d2fb7d5f6c9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3beb4b845fcc4fe78ba6d2fb7d5f6c9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33875 } } }
I20260619 02:43:45.965271 26905 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3beb4b845fcc4fe78ba6d2fb7d5f6c9b [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:45.965410 26906 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3beb4b845fcc4fe78ba6d2fb7d5f6c9b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3beb4b845fcc4fe78ba6d2fb7d5f6c9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3beb4b845fcc4fe78ba6d2fb7d5f6c9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33875 } } }
I20260619 02:43:45.965464 26906 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3beb4b845fcc4fe78ba6d2fb7d5f6c9b [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:45.965767 26912 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:43:45.966389 26912 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:43:45.967941 26904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:45.966475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43109 (local address 127.0.0.1:33875)
0619 02:43:45.966513 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:45.966515 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:45.966942 (+ 427us) server_negotiation.cc:407] Connection header received
0619 02:43:45.967014 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:45.967016 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:45.967034 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:45.967095 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:45.967701 (+ 606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:45.967702 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:45.967703 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:43:45.967707 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:45.967720 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:43:45.967724 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:45.967769 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:45.967867 (+ 98us) server_negotiation.cc:299] Negotiation successful
0619 02:43:45.967884 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 02:43:45.968254 26912 catalog_manager.cc:1347] Generated new cluster ID: f9a7de147d154d94ae3f24addf8ab404
I20260619 02:43:45.968338 26912 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:43:45.973181 26912 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:43:45.973352 26912 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:43:45.985818 26912 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3beb4b845fcc4fe78ba6d2fb7d5f6c9b: Generated new TSK 0
I20260619 02:43:45.986020 26912 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:43:45.993419 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 26832
2026-06-19T02:43:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1320 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836772852275-6085-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:43:46 dist-test-slave-zpbg krb5kdc[26931](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:43:46 dist-test-slave-zpbg krb5kdc[26931](info): set up 2 sockets
Jun 19 02:43:46 dist-test-slave-zpbg krb5kdc[26931](info): commencing operation
krb5kdc: starting...
W20260619 02:43:47.030969 6085 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Jun 19 02:43:47 dist-test-slave-zpbg krb5kdc[26931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837027, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:43:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:43:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836772852275-6085-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836772852275-6085-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:43:47.166105 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34471
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836772852275-6085-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34471
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:43:47.248685 26944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:47.248896 26944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:47.248920 26944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:47.250015 26944 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:43:47.250051 26944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:47.250064 26944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:47.250077 26944 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:43:47.250089 26944 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:43:47.252044 26944 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836772852275-6085-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34471
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836772852275-6085-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26944
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:43:47.252246 26944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:47.252413 26944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:47.257486 26949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:47.257542 26950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:47.257658 26952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:47.257562 26944 server_base.cc:1034] running on GCE node
I20260619 02:43:47.257946 26944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:47.258260 26944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:47.259428 26944 hybrid_clock.cc:648] HybridClock initialized: now 1781837027259405 us; error 42 us; skew 500 ppm
Jun 19 02:43:47 dist-test-slave-zpbg krb5kdc[26931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837027, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:47.262004 26944 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:43:47.262454 26944 webserver.cc:460] Webserver started at http://127.0.0.1:33361/ using document root <none> and password file <none>
I20260619 02:43:47.262720 26944 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:47.262818 26944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:47.263000 26944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:43:47.264098 26944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836772852275-6085-0/minicluster-data/master-0/data/instance:
uuid: "bb0bcc8ad03741358cb440208438e39e"
format_stamp: "Formatted at 2026-06-19 02:43:47 on dist-test-slave-zpbg"
server_key: "48fe55587725ae260081c414eaa0b98d"
server_key_iv: "42ce65f07d2312b63c60673e43c15aa9"
server_key_version: "encryptionkey@0"
I20260619 02:43:47.264472 26944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836772852275-6085-0/minicluster-data/master-0/wal/instance:
uuid: "bb0bcc8ad03741358cb440208438e39e"
format_stamp: "Formatted at 2026-06-19 02:43:47 on dist-test-slave-zpbg"
server_key: "48fe55587725ae260081c414eaa0b98d"
server_key_iv: "42ce65f07d2312b63c60673e43c15aa9"
server_key_version: "encryptionkey@0"
I20260619 02:43:47.265862 26944 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:43:47.266670 26959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:47.266881 26944 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:43:47.266994 26944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "bb0bcc8ad03741358cb440208438e39e"
format_stamp: "Formatted at 2026-06-19 02:43:47 on dist-test-slave-zpbg"
server_key: "48fe55587725ae260081c414eaa0b98d"
server_key_iv: "42ce65f07d2312b63c60673e43c15aa9"
server_key_version: "encryptionkey@0"
I20260619 02:43:47.267100 26944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:47.276636 26944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:47.279167 26944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:47.279336 26944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:47.283362 26944 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34471
I20260619 02:43:47.283416 27011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34471 every 8 connection(s)
I20260619 02:43:47.283787 26944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:43:47.284835 27012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:47.287177 27012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb0bcc8ad03741358cb440208438e39e: Bootstrap starting.
I20260619 02:43:47.287701 27012 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb0bcc8ad03741358cb440208438e39e: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:47.287926 27012 log.cc:824] T 00000000000000000000000000000000 P bb0bcc8ad03741358cb440208438e39e: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:47.288617 27012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb0bcc8ad03741358cb440208438e39e: No bootstrap required, opened a new log
I20260619 02:43:47.290189 27012 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb0bcc8ad03741358cb440208438e39e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb0bcc8ad03741358cb440208438e39e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34471 } }
I20260619 02:43:47.290338 27012 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb0bcc8ad03741358cb440208438e39e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:47.290361 27012 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb0bcc8ad03741358cb440208438e39e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb0bcc8ad03741358cb440208438e39e, State: Initialized, Role: FOLLOWER
I20260619 02:43:47.290516 27012 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb0bcc8ad03741358cb440208438e39e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb0bcc8ad03741358cb440208438e39e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34471 } }
I20260619 02:43:47.290601 27012 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb0bcc8ad03741358cb440208438e39e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:43:47.290623 27012 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb0bcc8ad03741358cb440208438e39e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:43:47.290653 27012 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb0bcc8ad03741358cb440208438e39e [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:47.291246 27012 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb0bcc8ad03741358cb440208438e39e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb0bcc8ad03741358cb440208438e39e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34471 } }
I20260619 02:43:47.291334 27012 leader_election.cc:302] T 00000000000000000000000000000000 P bb0bcc8ad03741358cb440208438e39e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb0bcc8ad03741358cb440208438e39e; no voters:
I20260619 02:43:47.291460 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 26944
I20260619 02:43:47.291507 27012 leader_election.cc:288] T 00000000000000000000000000000000 P bb0bcc8ad03741358cb440208438e39e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:43:47.291546 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836772852275-6085-0/minicluster-data/master-0/wal/instance
I20260619 02:43:47.291533 27015 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb0bcc8ad03741358cb440208438e39e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:43:47.291648 27015 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb0bcc8ad03741358cb440208438e39e [term 1 LEADER]: Becoming Leader. State: Replica: bb0bcc8ad03741358cb440208438e39e, State: Running, Role: LEADER
I20260619 02:43:47.291729 27015 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb0bcc8ad03741358cb440208438e39e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb0bcc8ad03741358cb440208438e39e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34471 } }
I20260619 02:43:47.291766 6085 external_mini_cluster.cc:1428] Setting key 62d47f725d0f840c2aabee3ec08a93a7
I20260619 02:43:47.291776 27012 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb0bcc8ad03741358cb440208438e39e [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:43:47.292034 27015 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb0bcc8ad03741358cb440208438e39e [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb0bcc8ad03741358cb440208438e39e. Latest consensus state: current_term: 1 leader_uuid: "bb0bcc8ad03741358cb440208438e39e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb0bcc8ad03741358cb440208438e39e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34471 } } }
I20260619 02:43:47.292105 27015 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb0bcc8ad03741358cb440208438e39e [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:47.292338 27016 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb0bcc8ad03741358cb440208438e39e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb0bcc8ad03741358cb440208438e39e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb0bcc8ad03741358cb440208438e39e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34471 } } }
I20260619 02:43:47.292472 27016 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb0bcc8ad03741358cb440208438e39e [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:47.292809 27020 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:43:47.293356 27020 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:43:47.295106 27020 catalog_manager.cc:1347] Generated new cluster ID: e6027f38dd74442ca53dc945839289b5
I20260619 02:43:47.295159 27020 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 02:43:47 dist-test-slave-zpbg krb5kdc[26931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837027, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 02:43:47.304126 27026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:47.292445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.226:57735 (local address 127.0.0.1:34471)
0619 02:43:47.292630 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:47.292639 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:47.292677 (+ 38us) server_negotiation.cc:407] Connection header received
0619 02:43:47.292805 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:47.292813 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:47.292921 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:47.293072 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:47.293954 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:47.295360 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:47.296431 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:47.297070 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:47.297183 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:47.300088 (+ 2905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:47.300092 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:47.300093 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:47.300105 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:47.302161 (+ 2056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:47.302650 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:47.302654 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:47.302654 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:47.302793 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:47.303038 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:47.303039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:47.303039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:47.303375 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:47.303560 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:47.303668 (+ 108us) server_negotiation.cc:299] Negotiation successful
0619 02:43:47.303884 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":77,"threads_started":1}
I20260619 02:43:47.305150 6085 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:43:47.313246 27020 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:43:47.313964 27020 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:43:47.314087 27026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:47.306570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.226:56507 (local address 127.0.0.1:34471)
0619 02:43:47.306616 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:47.306618 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:47.306745 (+ 127us) server_negotiation.cc:407] Connection header received
0619 02:43:47.306797 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:47.306797 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:47.306808 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:47.306873 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:47.307431 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:47.308237 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:47.309202 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:47.309529 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:47.309607 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:47.310949 (+ 1342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:47.310951 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:47.310951 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:47.310958 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:47.312564 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:47.313015 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:47.313016 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:47.313016 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:47.313072 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:47.313320 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:47.313321 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:47.313322 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:47.313432 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:47.313534 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:47.313758 (+ 224us) server_negotiation.cc:299] Negotiation successful
0619 02:43:47.313908 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":68,"server-negotiator.queue_time_us":28}
I20260619 02:43:47.322168 27020 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bb0bcc8ad03741358cb440208438e39e: Generated new TSK 0
I20260619 02:43:47.322412 27020 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:43:47.328326 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 26944
W20260619 02:43:47.332556 26929 connection.cc:535] client connection to 127.0.0.1:34471 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34471): Connection reset by peer (error 104)
2026-06-19T02:43:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1334 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836772852275-6085-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:43:47 dist-test-slave-zpbg krb5kdc[27043](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:43:47 dist-test-slave-zpbg krb5kdc[27043](info): set up 2 sockets
Jun 19 02:43:47 dist-test-slave-zpbg krb5kdc[27043](info): commencing operation
krb5kdc: starting...
W20260619 02:43:48.362934 6085 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Jun 19 02:43:48 dist-test-slave-zpbg krb5kdc[27043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837028, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:43:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:43:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836772852275-6085-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836772852275-6085-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:43:48.483964 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34903
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836772852275-6085-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34903
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:43:48.564783 27056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:48.565029 27056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:48.565065 27056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:48.566177 27056 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:43:48.566253 27056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:48.566280 27056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:48.566313 27056 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:43:48.566335 27056 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:43:48.567770 27056 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836772852275-6085-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34903
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836772852275-6085-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27056
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:43:48.568049 27056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:48.568276 27056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:48.573120 27061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:48.573127 27064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:48.573133 27062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:48.573201 27056 server_base.cc:1034] running on GCE node
I20260619 02:43:48.573586 27056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:48.573884 27056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:48.575059 27056 hybrid_clock.cc:648] HybridClock initialized: now 1781837028575036 us; error 48 us; skew 500 ppm
Jun 19 02:43:48 dist-test-slave-zpbg krb5kdc[27043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837028, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:48.581590 27056 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:43:48.582065 27056 webserver.cc:460] Webserver started at http://127.0.0.1:40779/ using document root <none> and password file <none>
I20260619 02:43:48.582294 27056 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:48.582337 27056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:48.582428 27056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:43:48.583495 27056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836772852275-6085-0/minicluster-data/master-0/data/instance:
uuid: "707710d5bdba4eebaf4bbffdd9607e6e"
format_stamp: "Formatted at 2026-06-19 02:43:48 on dist-test-slave-zpbg"
server_key: "a655ada86b786b1a0be07b4c80a88264"
server_key_iv: "1c869b876d5eeee15eec8d416170fb80"
server_key_version: "encryptionkey@0"
I20260619 02:43:48.583789 27056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836772852275-6085-0/minicluster-data/master-0/wal/instance:
uuid: "707710d5bdba4eebaf4bbffdd9607e6e"
format_stamp: "Formatted at 2026-06-19 02:43:48 on dist-test-slave-zpbg"
server_key: "a655ada86b786b1a0be07b4c80a88264"
server_key_iv: "1c869b876d5eeee15eec8d416170fb80"
server_key_version: "encryptionkey@0"
I20260619 02:43:48.584996 27056 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:43:48.585685 27071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:48.585886 27056 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:43:48.585958 27056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "707710d5bdba4eebaf4bbffdd9607e6e"
format_stamp: "Formatted at 2026-06-19 02:43:48 on dist-test-slave-zpbg"
server_key: "a655ada86b786b1a0be07b4c80a88264"
server_key_iv: "1c869b876d5eeee15eec8d416170fb80"
server_key_version: "encryptionkey@0"
I20260619 02:43:48.586009 27056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:48.628389 27056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:48.630740 27056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:48.630924 27056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:48.634753 27056 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34903
I20260619 02:43:48.634811 27123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34903 every 8 connection(s)
I20260619 02:43:48.635227 27056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:43:48.636119 27124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:48.638512 27124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 707710d5bdba4eebaf4bbffdd9607e6e: Bootstrap starting.
I20260619 02:43:48.639021 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 27056
I20260619 02:43:48.639120 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836772852275-6085-0/minicluster-data/master-0/wal/instance
I20260619 02:43:48.639261 27124 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 707710d5bdba4eebaf4bbffdd9607e6e: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:48.639400 6085 external_mini_cluster.cc:1428] Setting key 8c7f87824152413021ca5166aa82a84e
I20260619 02:43:48.639673 27124 log.cc:824] T 00000000000000000000000000000000 P 707710d5bdba4eebaf4bbffdd9607e6e: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:48.640478 27124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 707710d5bdba4eebaf4bbffdd9607e6e: No bootstrap required, opened a new log
I20260619 02:43:48.642731 27124 raft_consensus.cc:348] T 00000000000000000000000000000000 P 707710d5bdba4eebaf4bbffdd9607e6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "707710d5bdba4eebaf4bbffdd9607e6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34903 } }
I20260619 02:43:48.642936 27124 raft_consensus.cc:374] T 00000000000000000000000000000000 P 707710d5bdba4eebaf4bbffdd9607e6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:48.642969 27124 raft_consensus.cc:729] T 00000000000000000000000000000000 P 707710d5bdba4eebaf4bbffdd9607e6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 707710d5bdba4eebaf4bbffdd9607e6e, State: Initialized, Role: FOLLOWER
I20260619 02:43:48.643085 27124 consensus_queue.cc:260] T 00000000000000000000000000000000 P 707710d5bdba4eebaf4bbffdd9607e6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "707710d5bdba4eebaf4bbffdd9607e6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34903 } }
I20260619 02:43:48.643155 27124 raft_consensus.cc:388] T 00000000000000000000000000000000 P 707710d5bdba4eebaf4bbffdd9607e6e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:43:48.643183 27124 raft_consensus.cc:482] T 00000000000000000000000000000000 P 707710d5bdba4eebaf4bbffdd9607e6e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:43:48.643224 27124 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 707710d5bdba4eebaf4bbffdd9607e6e [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:48.643801 27124 raft_consensus.cc:504] T 00000000000000000000000000000000 P 707710d5bdba4eebaf4bbffdd9607e6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "707710d5bdba4eebaf4bbffdd9607e6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34903 } }
I20260619 02:43:48.643908 27124 leader_election.cc:302] T 00000000000000000000000000000000 P 707710d5bdba4eebaf4bbffdd9607e6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 707710d5bdba4eebaf4bbffdd9607e6e; no voters:
I20260619 02:43:48.644083 27124 leader_election.cc:288] T 00000000000000000000000000000000 P 707710d5bdba4eebaf4bbffdd9607e6e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:43:48.644268 27130 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 707710d5bdba4eebaf4bbffdd9607e6e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:43:48.644356 27124 sys_catalog.cc:564] T 00000000000000000000000000000000 P 707710d5bdba4eebaf4bbffdd9607e6e [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:43:48.644508 27130 raft_consensus.cc:686] T 00000000000000000000000000000000 P 707710d5bdba4eebaf4bbffdd9607e6e [term 1 LEADER]: Becoming Leader. State: Replica: 707710d5bdba4eebaf4bbffdd9607e6e, State: Running, Role: LEADER
I20260619 02:43:48.644636 27130 consensus_queue.cc:237] T 00000000000000000000000000000000 P 707710d5bdba4eebaf4bbffdd9607e6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "707710d5bdba4eebaf4bbffdd9607e6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34903 } }
I20260619 02:43:48.645452 27128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:48.639987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.226:39903 (local address 127.0.0.1:34903)
0619 02:43:48.640206 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:48.640216 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:48.640257 (+ 41us) server_negotiation.cc:407] Connection header received
0619 02:43:48.640780 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:48.640787 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:48.640839 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:48.640961 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:48.641189 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:48.642116 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:48.643279 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:48.643713 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:48.643802 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:48.644607 (+ 805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:48.644610 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:48.644611 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:43:48.644823 (+ 212us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:48.644866 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:43:48.644884 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:48.644972 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:48.645193 (+ 221us) server_negotiation.cc:299] Negotiation successful
0619 02:43:48.645221 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":85,"threads_started":1}
I20260619 02:43:48.645874 27129 sys_catalog.cc:455] T 00000000000000000000000000000000 P 707710d5bdba4eebaf4bbffdd9607e6e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "707710d5bdba4eebaf4bbffdd9607e6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "707710d5bdba4eebaf4bbffdd9607e6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34903 } } }
I20260619 02:43:48.645959 27129 sys_catalog.cc:458] T 00000000000000000000000000000000 P 707710d5bdba4eebaf4bbffdd9607e6e [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:48.645831 27131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 707710d5bdba4eebaf4bbffdd9607e6e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 707710d5bdba4eebaf4bbffdd9607e6e. Latest consensus state: current_term: 1 leader_uuid: "707710d5bdba4eebaf4bbffdd9607e6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "707710d5bdba4eebaf4bbffdd9607e6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34903 } } }
I20260619 02:43:48.646085 27131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 707710d5bdba4eebaf4bbffdd9607e6e [sys.catalog]: This master's current role is: LEADER
W20260619 02:43:48.646327 27144 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 707710d5bdba4eebaf4bbffdd9607e6e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:43:48.646376 27144 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:43:48.646435 27146 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:43:48.646608 27146 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:43:48.646606 6085 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:43:48.648279 27146 catalog_manager.cc:1347] Generated new cluster ID: 0cf03660a46d4ff18c20e1dae8da95e4
I20260619 02:43:48.648330 27146 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:43:48.651103 27128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:48.648029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.226:49501 (local address 127.0.0.1:34903)
0619 02:43:48.648063 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:48.648065 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:48.648230 (+ 165us) server_negotiation.cc:407] Connection header received
0619 02:43:48.648330 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:48.648331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:48.648341 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:48.648389 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:48.648516 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:48.648985 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:48.649879 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:48.650100 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:48.650189 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:48.650863 (+ 674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:48.650864 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:48.650865 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:43:48.650869 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:48.650883 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:43:48.650887 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:48.650941 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:48.650999 (+ 58us) server_negotiation.cc:299] Negotiation successful
0619 02:43:48.651010 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 02:43:48.664008 27146 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:43:48.664592 27146 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:43:48.668751 27146 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 707710d5bdba4eebaf4bbffdd9607e6e: Generated new TSK 0
I20260619 02:43:48.668941 27146 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:43:48.672349 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 27056
W20260619 02:43:48.676519 27041 connection.cc:535] client connection to 127.0.0.1:34903 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34903): Connection reset by peer (error 104)
2026-06-19T02:43:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1343 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836772852275-6085-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:43:48 dist-test-slave-zpbg krb5kdc[27155](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:43:48 dist-test-slave-zpbg krb5kdc[27155](info): set up 2 sockets
Jun 19 02:43:48 dist-test-slave-zpbg krb5kdc[27155](info): commencing operation
krb5kdc: starting...
W20260619 02:43:49.710906 6085 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Jun 19 02:43:49 dist-test-slave-zpbg krb5kdc[27155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837029, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:43:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:43:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836772852275-6085-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836772852275-6085-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:43:49.829870 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33771
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836772852275-6085-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33771
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:43:49.912609 27168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:49.912820 27168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:49.912843 27168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:49.913977 27168 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:43:49.914016 27168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:49.914031 27168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:49.914043 27168 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:43:49.914057 27168 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:43:49.915486 27168 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836772852275-6085-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33771
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836772852275-6085-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27168
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:43:49.915746 27168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:49.916009 27168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:49.920967 27176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:49.920974 27173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:49.920974 27174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:49.921250 27168 server_base.cc:1034] running on GCE node
I20260619 02:43:49.921479 27168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:49.921770 27168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:49.922914 27168 hybrid_clock.cc:648] HybridClock initialized: now 1781837029922906 us; error 40 us; skew 500 ppm
Jun 19 02:43:49 dist-test-slave-zpbg krb5kdc[27155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837029, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:49.925505 27168 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:43:49.926007 27168 webserver.cc:460] Webserver started at http://127.0.0.1:44025/ using document root <none> and password file <none>
I20260619 02:43:49.926247 27168 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:49.926321 27168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:49.926478 27168 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:43:49.927639 27168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836772852275-6085-0/minicluster-data/master-0/data/instance:
uuid: "9d86c05eee6c4a569adc7c1696cb73a6"
format_stamp: "Formatted at 2026-06-19 02:43:49 on dist-test-slave-zpbg"
server_key: "385ce35ecc38e25b92db3d6c5c730ae2"
server_key_iv: "7a4ef841d11b9fcbe9775885cc4c7141"
server_key_version: "encryptionkey@0"
I20260619 02:43:49.928011 27168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836772852275-6085-0/minicluster-data/master-0/wal/instance:
uuid: "9d86c05eee6c4a569adc7c1696cb73a6"
format_stamp: "Formatted at 2026-06-19 02:43:49 on dist-test-slave-zpbg"
server_key: "385ce35ecc38e25b92db3d6c5c730ae2"
server_key_iv: "7a4ef841d11b9fcbe9775885cc4c7141"
server_key_version: "encryptionkey@0"
I20260619 02:43:49.929363 27168 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 02:43:49.930143 27183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:49.930328 27168 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:43:49.930431 27168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "9d86c05eee6c4a569adc7c1696cb73a6"
format_stamp: "Formatted at 2026-06-19 02:43:49 on dist-test-slave-zpbg"
server_key: "385ce35ecc38e25b92db3d6c5c730ae2"
server_key_iv: "7a4ef841d11b9fcbe9775885cc4c7141"
server_key_version: "encryptionkey@0"
I20260619 02:43:49.930514 27168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:49.933934 27168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:49.936198 27168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:49.936365 27168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:49.940563 27168 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33771
I20260619 02:43:49.940593 27235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33771 every 8 connection(s)
I20260619 02:43:49.940939 27168 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:43:49.941882 27236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:49.944180 27236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d86c05eee6c4a569adc7c1696cb73a6: Bootstrap starting.
I20260619 02:43:49.944759 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 27168
I20260619 02:43:49.944792 27236 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d86c05eee6c4a569adc7c1696cb73a6: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:49.944876 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836772852275-6085-0/minicluster-data/master-0/wal/instance
I20260619 02:43:49.945078 6085 external_mini_cluster.cc:1428] Setting key 1276c974e612c871b8f11746765920c8
I20260619 02:43:49.945104 27236 log.cc:824] T 00000000000000000000000000000000 P 9d86c05eee6c4a569adc7c1696cb73a6: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:49.945930 27236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d86c05eee6c4a569adc7c1696cb73a6: No bootstrap required, opened a new log
I20260619 02:43:49.947726 27236 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d86c05eee6c4a569adc7c1696cb73a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d86c05eee6c4a569adc7c1696cb73a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33771 } }
I20260619 02:43:49.947890 27236 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9d86c05eee6c4a569adc7c1696cb73a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:49.947937 27236 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d86c05eee6c4a569adc7c1696cb73a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d86c05eee6c4a569adc7c1696cb73a6, State: Initialized, Role: FOLLOWER
I20260619 02:43:49.948078 27236 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d86c05eee6c4a569adc7c1696cb73a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d86c05eee6c4a569adc7c1696cb73a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33771 } }
I20260619 02:43:49.948182 27236 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9d86c05eee6c4a569adc7c1696cb73a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:43:49.948218 27236 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d86c05eee6c4a569adc7c1696cb73a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:43:49.948266 27236 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d86c05eee6c4a569adc7c1696cb73a6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:49.949075 27240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:49.945585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.226:34243 (local address 127.0.0.1:33771)
0619 02:43:49.945782 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:49.945792 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:49.946165 (+ 373us) server_negotiation.cc:407] Connection header received
0619 02:43:49.947733 (+ 1568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:49.947741 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:49.947773 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:49.947948 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:49.948656 (+ 708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:49.948659 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:49.948659 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:43:49.948663 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:49.948699 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:43:49.948710 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:49.948795 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:49.948865 (+ 70us) server_negotiation.cc:299] Negotiation successful
0619 02:43:49.948909 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":68,"threads_started":1}
I20260619 02:43:49.949708 27236 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d86c05eee6c4a569adc7c1696cb73a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d86c05eee6c4a569adc7c1696cb73a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33771 } }
I20260619 02:43:49.949836 27236 leader_election.cc:302] T 00000000000000000000000000000000 P 9d86c05eee6c4a569adc7c1696cb73a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d86c05eee6c4a569adc7c1696cb73a6; no voters:
I20260619 02:43:49.950024 27236 leader_election.cc:288] T 00000000000000000000000000000000 P 9d86c05eee6c4a569adc7c1696cb73a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:43:49.950075 27241 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d86c05eee6c4a569adc7c1696cb73a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:43:49.950260 27241 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d86c05eee6c4a569adc7c1696cb73a6 [term 1 LEADER]: Becoming Leader. State: Replica: 9d86c05eee6c4a569adc7c1696cb73a6, State: Running, Role: LEADER
I20260619 02:43:49.950366 27236 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d86c05eee6c4a569adc7c1696cb73a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:43:49.950361 6085 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:43:49.950402 27241 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d86c05eee6c4a569adc7c1696cb73a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d86c05eee6c4a569adc7c1696cb73a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33771 } }
I20260619 02:43:49.950723 27241 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d86c05eee6c4a569adc7c1696cb73a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d86c05eee6c4a569adc7c1696cb73a6. Latest consensus state: current_term: 1 leader_uuid: "9d86c05eee6c4a569adc7c1696cb73a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d86c05eee6c4a569adc7c1696cb73a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33771 } } }
I20260619 02:43:49.950737 27242 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d86c05eee6c4a569adc7c1696cb73a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9d86c05eee6c4a569adc7c1696cb73a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d86c05eee6c4a569adc7c1696cb73a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33771 } } }
I20260619 02:43:49.950842 27241 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d86c05eee6c4a569adc7c1696cb73a6 [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:49.951043 27242 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d86c05eee6c4a569adc7c1696cb73a6 [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:49.951325 27246 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:43:49.952075 27246 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:43:49.953629 27246 catalog_manager.cc:1347] Generated new cluster ID: dba3ebcee3594bd8b393e4b7c3a223d6
I20260619 02:43:49.953681 27246 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:43:49.954815 27240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:49.953407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.226:56485 (local address 127.0.0.1:33771)
0619 02:43:49.953449 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:49.953451 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:49.953647 (+ 196us) server_negotiation.cc:407] Connection header received
0619 02:43:49.953755 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:49.953756 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:49.953773 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:49.953814 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:49.954539 (+ 725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:49.954540 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:49.954541 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:43:49.954546 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:49.954561 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:43:49.954566 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:49.954630 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:49.954693 (+ 63us) server_negotiation.cc:299] Negotiation successful
0619 02:43:49.954708 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:43:49.977291 27246 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:43:49.977489 27246 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:43:49.993819 27246 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9d86c05eee6c4a569adc7c1696cb73a6: Generated new TSK 0
I20260619 02:43:49.994082 27246 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:43:50.007078 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 27168
2026-06-19T02:43:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1335 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836772852275-6085-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:43:50 dist-test-slave-zpbg krb5kdc[27267](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:43:50 dist-test-slave-zpbg krb5kdc[27267](info): set up 2 sockets
Jun 19 02:43:50 dist-test-slave-zpbg krb5kdc[27267](info): commencing operation
krb5kdc: starting...
W20260619 02:43:51.038945 6085 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Jun 19 02:43:51 dist-test-slave-zpbg krb5kdc[27267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837031, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:43:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:43:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836772852275-6085-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836772852275-6085-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:43:51.157951 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39245
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836772852275-6085-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39245
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:43:51.238379 27280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:51.238605 27280 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:51.238628 27280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:51.239774 27280 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:43:51.239815 27280 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:51.239830 27280 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:51.239842 27280 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:43:51.239856 27280 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:43:51.241282 27280 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836772852275-6085-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39245
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836772852275-6085-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27280
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:43:51.241478 27280 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:51.241652 27280 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:43:51.246838 27280 server_base.cc:1034] running on GCE node
W20260619 02:43:51.246817 27286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:51.246789 27285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:51.246814 27288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:51.247304 27280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:51.247603 27280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:51.248772 27280 hybrid_clock.cc:648] HybridClock initialized: now 1781837031248748 us; error 42 us; skew 500 ppm
Jun 19 02:43:51 dist-test-slave-zpbg krb5kdc[27267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837031, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:51.251543 27280 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:43:51.252056 27280 webserver.cc:460] Webserver started at http://127.0.0.1:45989/ using document root <none> and password file <none>
I20260619 02:43:51.252314 27280 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:51.252384 27280 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:51.252542 27280 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:43:51.253634 27280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836772852275-6085-0/minicluster-data/master-0/data/instance:
uuid: "97ad0832f88f4906bad04e3cea23b92b"
format_stamp: "Formatted at 2026-06-19 02:43:51 on dist-test-slave-zpbg"
server_key: "443cf710219f67999cd8e68983b95ff9"
server_key_iv: "c39a0725388f1ed0b67b44c36f79fbc2"
server_key_version: "encryptionkey@0"
I20260619 02:43:51.254010 27280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836772852275-6085-0/minicluster-data/master-0/wal/instance:
uuid: "97ad0832f88f4906bad04e3cea23b92b"
format_stamp: "Formatted at 2026-06-19 02:43:51 on dist-test-slave-zpbg"
server_key: "443cf710219f67999cd8e68983b95ff9"
server_key_iv: "c39a0725388f1ed0b67b44c36f79fbc2"
server_key_version: "encryptionkey@0"
I20260619 02:43:51.255548 27280 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:43:51.256336 27295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:51.256511 27280 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:43:51.256610 27280 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "97ad0832f88f4906bad04e3cea23b92b"
format_stamp: "Formatted at 2026-06-19 02:43:51 on dist-test-slave-zpbg"
server_key: "443cf710219f67999cd8e68983b95ff9"
server_key_iv: "c39a0725388f1ed0b67b44c36f79fbc2"
server_key_version: "encryptionkey@0"
I20260619 02:43:51.256713 27280 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:51.277856 27280 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:51.280076 27280 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:51.280246 27280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:51.284413 27280 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39245
I20260619 02:43:51.284458 27347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39245 every 8 connection(s)
I20260619 02:43:51.284850 27280 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:43:51.285858 27348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:51.288101 27348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97ad0832f88f4906bad04e3cea23b92b: Bootstrap starting.
I20260619 02:43:51.288628 27348 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97ad0832f88f4906bad04e3cea23b92b: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:51.288861 27348 log.cc:824] T 00000000000000000000000000000000 P 97ad0832f88f4906bad04e3cea23b92b: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:51.289552 27348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97ad0832f88f4906bad04e3cea23b92b: No bootstrap required, opened a new log
I20260619 02:43:51.291231 27348 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97ad0832f88f4906bad04e3cea23b92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97ad0832f88f4906bad04e3cea23b92b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39245 } }
I20260619 02:43:51.291373 27348 raft_consensus.cc:374] T 00000000000000000000000000000000 P 97ad0832f88f4906bad04e3cea23b92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:51.291395 27348 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97ad0832f88f4906bad04e3cea23b92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97ad0832f88f4906bad04e3cea23b92b, State: Initialized, Role: FOLLOWER
I20260619 02:43:51.291538 27348 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97ad0832f88f4906bad04e3cea23b92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97ad0832f88f4906bad04e3cea23b92b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39245 } }
I20260619 02:43:51.291646 27348 raft_consensus.cc:388] T 00000000000000000000000000000000 P 97ad0832f88f4906bad04e3cea23b92b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:43:51.291671 27348 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97ad0832f88f4906bad04e3cea23b92b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:43:51.291702 27348 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97ad0832f88f4906bad04e3cea23b92b [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:51.292214 27348 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97ad0832f88f4906bad04e3cea23b92b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97ad0832f88f4906bad04e3cea23b92b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39245 } }
I20260619 02:43:51.292296 27348 leader_election.cc:302] T 00000000000000000000000000000000 P 97ad0832f88f4906bad04e3cea23b92b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97ad0832f88f4906bad04e3cea23b92b; no voters:
I20260619 02:43:51.292420 27348 leader_election.cc:288] T 00000000000000000000000000000000 P 97ad0832f88f4906bad04e3cea23b92b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:43:51.292483 27351 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97ad0832f88f4906bad04e3cea23b92b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:43:51.292681 27351 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97ad0832f88f4906bad04e3cea23b92b [term 1 LEADER]: Becoming Leader. State: Replica: 97ad0832f88f4906bad04e3cea23b92b, State: Running, Role: LEADER
I20260619 02:43:51.292754 27348 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97ad0832f88f4906bad04e3cea23b92b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:43:51.293044 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 27280
I20260619 02:43:51.293195 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836772852275-6085-0/minicluster-data/master-0/wal/instance
I20260619 02:43:51.293395 6085 external_mini_cluster.cc:1428] Setting key 6e16dd3a0bb54db3b6f2cca3a99375d3
I20260619 02:43:51.292843 27351 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97ad0832f88f4906bad04e3cea23b92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97ad0832f88f4906bad04e3cea23b92b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39245 } }
I20260619 02:43:51.294116 27353 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97ad0832f88f4906bad04e3cea23b92b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97ad0832f88f4906bad04e3cea23b92b. Latest consensus state: current_term: 1 leader_uuid: "97ad0832f88f4906bad04e3cea23b92b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97ad0832f88f4906bad04e3cea23b92b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39245 } } }
I20260619 02:43:51.294144 27352 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97ad0832f88f4906bad04e3cea23b92b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "97ad0832f88f4906bad04e3cea23b92b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97ad0832f88f4906bad04e3cea23b92b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39245 } } }
I20260619 02:43:51.294454 27353 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97ad0832f88f4906bad04e3cea23b92b [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:51.294461 27352 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97ad0832f88f4906bad04e3cea23b92b [sys.catalog]: This master's current role is: LEADER
W20260619 02:43:51.294921 27367 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 97ad0832f88f4906bad04e3cea23b92b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:43:51.294975 27367 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:43:51.295054 27368 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:43:51.295269 27368 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:43:51.296978 27368 catalog_manager.cc:1347] Generated new cluster ID: 955b4d7e2a8a46fd9662d11c676fac3d
I20260619 02:43:51.297037 27368 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 02:43:51 dist-test-slave-zpbg krb5kdc[27267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837031, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 02:43:51.305245 27365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:51.293932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.226:38473 (local address 127.0.0.1:39245)
0619 02:43:51.294294 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:51.294305 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:51.294337 (+ 32us) server_negotiation.cc:407] Connection header received
0619 02:43:51.294463 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:51.294471 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:51.294533 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:51.294710 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:51.295523 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:51.296686 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:51.297799 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:51.298451 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:51.298587 (+ 136us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:51.301398 (+ 2811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:51.301402 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:51.301402 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:51.301415 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:51.303477 (+ 2062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:51.303946 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:51.303949 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:51.303949 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:51.304067 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:51.304316 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:51.304317 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:51.304317 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:51.304629 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:51.304787 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:51.304936 (+ 149us) server_negotiation.cc:299] Negotiation successful
0619 02:43:51.305035 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":78,"threads_started":1}
I20260619 02:43:51.306327 6085 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:43:51.312073 27368 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:43:51.312728 27368 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:43:51.315433 27365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:51.307862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.226:33139 (local address 127.0.0.1:39245)
0619 02:43:51.307897 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:51.307899 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:51.307908 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:43:51.307929 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:51.307930 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:51.307940 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:51.308004 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:51.308573 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:51.309360 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:51.310338 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:51.310657 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:51.310731 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:51.312257 (+ 1526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:51.312258 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:51.312258 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:51.312265 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:51.314138 (+ 1873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:51.314566 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:51.314567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:51.314567 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:51.314629 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:51.314883 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:51.314883 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:51.314884 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:51.315014 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:51.315138 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:51.315272 (+ 134us) server_negotiation.cc:299] Negotiation successful
0619 02:43:51.315337 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 02:43:51.317762 27368 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 97ad0832f88f4906bad04e3cea23b92b: Generated new TSK 0
I20260619 02:43:51.317926 27368 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:43:51.320230 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 27280
W20260619 02:43:51.324172 27265 connection.cc:535] client connection to 127.0.0.1:39245 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39245): Connection reset by peer (error 104)
2026-06-19T02:43:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1312 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836772852275-6085-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:43:51 dist-test-slave-zpbg krb5kdc[27379](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:43:51 dist-test-slave-zpbg krb5kdc[27379](info): set up 2 sockets
Jun 19 02:43:51 dist-test-slave-zpbg krb5kdc[27379](info): commencing operation
krb5kdc: starting...
W20260619 02:43:52.359809 6085 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Jun 19 02:43:52 dist-test-slave-zpbg krb5kdc[27379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:43:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:43:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836772852275-6085-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836772852275-6085-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:43:52.477077 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33777
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836772852275-6085-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33777
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:43:52.556734 27392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:52.556933 27392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:52.556957 27392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:52.558219 27392 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:43:52.558295 27392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:52.558320 27392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:52.558352 27392 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:43:52.558379 27392 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:43:52.559938 27392 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836772852275-6085-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33777
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836772852275-6085-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27392
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:43:52.560225 27392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:52.560453 27392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:52.565362 27400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:52.565394 27398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:52.565559 27392 server_base.cc:1034] running on GCE node
W20260619 02:43:52.565394 27397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:52.565883 27392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:52.566208 27392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:52.567355 27392 hybrid_clock.cc:648] HybridClock initialized: now 1781837032567355 us; error 37 us; skew 500 ppm
Jun 19 02:43:52 dist-test-slave-zpbg krb5kdc[27379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837032, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:52.569795 27392 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:43:52.570236 27392 webserver.cc:460] Webserver started at http://127.0.0.1:34079/ using document root <none> and password file <none>
I20260619 02:43:52.570429 27392 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:52.570468 27392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:52.570555 27392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:43:52.571589 27392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836772852275-6085-0/minicluster-data/master-0/data/instance:
uuid: "3fe796eb59c54715b4c7dbc14726d812"
format_stamp: "Formatted at 2026-06-19 02:43:52 on dist-test-slave-zpbg"
server_key: "0bae4552093dc9ebf1bad1d5664f8e9e"
server_key_iv: "af31727c3164cd2c782fa6a975a749b1"
server_key_version: "encryptionkey@0"
I20260619 02:43:52.571874 27392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836772852275-6085-0/minicluster-data/master-0/wal/instance:
uuid: "3fe796eb59c54715b4c7dbc14726d812"
format_stamp: "Formatted at 2026-06-19 02:43:52 on dist-test-slave-zpbg"
server_key: "0bae4552093dc9ebf1bad1d5664f8e9e"
server_key_iv: "af31727c3164cd2c782fa6a975a749b1"
server_key_version: "encryptionkey@0"
I20260619 02:43:52.573112 27392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 02:43:52.573777 27407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:52.573926 27392 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:43:52.574023 27392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "3fe796eb59c54715b4c7dbc14726d812"
format_stamp: "Formatted at 2026-06-19 02:43:52 on dist-test-slave-zpbg"
server_key: "0bae4552093dc9ebf1bad1d5664f8e9e"
server_key_iv: "af31727c3164cd2c782fa6a975a749b1"
server_key_version: "encryptionkey@0"
I20260619 02:43:52.574088 27392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:52.588227 27392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:52.590039 27392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:52.590157 27392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:52.593979 27392 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33777
I20260619 02:43:52.594094 27459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33777 every 8 connection(s)
I20260619 02:43:52.594410 27392 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:43:52.595403 27460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:52.597625 27460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fe796eb59c54715b4c7dbc14726d812: Bootstrap starting.
I20260619 02:43:52.598193 27460 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3fe796eb59c54715b4c7dbc14726d812: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:52.598467 27460 log.cc:824] T 00000000000000000000000000000000 P 3fe796eb59c54715b4c7dbc14726d812: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:52.599234 27460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fe796eb59c54715b4c7dbc14726d812: No bootstrap required, opened a new log
I20260619 02:43:52.600809 27460 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3fe796eb59c54715b4c7dbc14726d812 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fe796eb59c54715b4c7dbc14726d812" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33777 } }
I20260619 02:43:52.600948 27460 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3fe796eb59c54715b4c7dbc14726d812 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:52.601011 27460 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3fe796eb59c54715b4c7dbc14726d812 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fe796eb59c54715b4c7dbc14726d812, State: Initialized, Role: FOLLOWER
I20260619 02:43:52.601176 27460 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3fe796eb59c54715b4c7dbc14726d812 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fe796eb59c54715b4c7dbc14726d812" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33777 } }
I20260619 02:43:52.601248 27460 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3fe796eb59c54715b4c7dbc14726d812 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:43:52.601287 27460 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3fe796eb59c54715b4c7dbc14726d812 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:43:52.601334 27460 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3fe796eb59c54715b4c7dbc14726d812 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:52.601502 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 27392
I20260619 02:43:52.601625 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836772852275-6085-0/minicluster-data/master-0/wal/instance
I20260619 02:43:52.601820 6085 external_mini_cluster.cc:1428] Setting key 21846f782317e3c1db90fbff4c65a4b4
I20260619 02:43:52.601980 27460 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3fe796eb59c54715b4c7dbc14726d812 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fe796eb59c54715b4c7dbc14726d812" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33777 } }
I20260619 02:43:52.602075 27460 leader_election.cc:302] T 00000000000000000000000000000000 P 3fe796eb59c54715b4c7dbc14726d812 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3fe796eb59c54715b4c7dbc14726d812; no voters:
I20260619 02:43:52.602303 27460 leader_election.cc:288] T 00000000000000000000000000000000 P 3fe796eb59c54715b4c7dbc14726d812 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:43:52.602344 27463 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3fe796eb59c54715b4c7dbc14726d812 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:43:52.602559 27463 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3fe796eb59c54715b4c7dbc14726d812 [term 1 LEADER]: Becoming Leader. State: Replica: 3fe796eb59c54715b4c7dbc14726d812, State: Running, Role: LEADER
I20260619 02:43:52.602625 27460 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3fe796eb59c54715b4c7dbc14726d812 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:43:52.602748 27463 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3fe796eb59c54715b4c7dbc14726d812 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fe796eb59c54715b4c7dbc14726d812" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33777 } }
I20260619 02:43:52.603230 27467 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fe796eb59c54715b4c7dbc14726d812 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3fe796eb59c54715b4c7dbc14726d812. Latest consensus state: current_term: 1 leader_uuid: "3fe796eb59c54715b4c7dbc14726d812" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fe796eb59c54715b4c7dbc14726d812" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33777 } } }
I20260619 02:43:52.603313 27466 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fe796eb59c54715b4c7dbc14726d812 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3fe796eb59c54715b4c7dbc14726d812" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fe796eb59c54715b4c7dbc14726d812" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33777 } } }
I20260619 02:43:52.603389 27466 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fe796eb59c54715b4c7dbc14726d812 [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:52.603339 27467 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fe796eb59c54715b4c7dbc14726d812 [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:52.603574 27474 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:43:52.604104 27474 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:43:52.605890 27474 catalog_manager.cc:1347] Generated new cluster ID: 49bfd8703d2240e397f254d8dd2aec6f
I20260619 02:43:52.605942 27474 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:43:52.608124 27465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:52.602321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.226:36115 (local address 127.0.0.1:33777)
0619 02:43:52.602684 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:52.602694 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:52.602728 (+ 34us) server_negotiation.cc:407] Connection header received
0619 02:43:52.602893 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:52.602901 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:52.602958 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:52.603103 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:52.603379 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:52.604886 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:52.605867 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:52.606428 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:52.606508 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:52.607428 (+ 920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:52.607432 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:52.607433 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:43:52.607643 (+ 210us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:52.607683 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:43:52.607698 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:52.607794 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:52.607887 (+ 93us) server_negotiation.cc:299] Negotiation successful
0619 02:43:52.607914 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":72,"threads_started":1}
I20260619 02:43:52.609063 6085 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:43:52.613914 27465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:52.610411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.226:44177 (local address 127.0.0.1:33777)
0619 02:43:52.610449 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:52.610451 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:52.610568 (+ 117us) server_negotiation.cc:407] Connection header received
0619 02:43:52.610598 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:52.610599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:52.610610 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:52.610678 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:52.610807 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:52.611618 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:52.612582 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:52.612912 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:52.612986 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:52.613653 (+ 667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:52.613655 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:52.613655 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:43:52.613660 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:52.613676 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:43:52.613680 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:52.613756 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:52.613830 (+ 74us) server_negotiation.cc:299] Negotiation successful
0619 02:43:52.613842 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":21}
I20260619 02:43:52.623028 27474 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:43:52.623661 27474 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:43:52.627664 27474 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3fe796eb59c54715b4c7dbc14726d812: Generated new TSK 0
I20260619 02:43:52.627849 27474 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:43:52.632624 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 27392
W20260619 02:43:52.636636 27377 connection.cc:535] client connection to 127.0.0.1:33777 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33777): Connection reset by peer (error 104)
2026-06-19T02:43:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1312 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836772852275-6085-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:43:52 dist-test-slave-zpbg krb5kdc[27491](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:43:52 dist-test-slave-zpbg krb5kdc[27491](info): set up 2 sockets
Jun 19 02:43:52 dist-test-slave-zpbg krb5kdc[27491](info): commencing operation
krb5kdc: starting...
W20260619 02:43:53.663012 6085 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Jun 19 02:43:53 dist-test-slave-zpbg krb5kdc[27491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837033, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:43:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:43:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836772852275-6085-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836772852275-6085-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:43:53.780979 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38619
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836772852275-6085-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38619
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:43:53.864140 27504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:53.864382 27504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:53.864423 27504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:53.865576 27504 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:43:53.865645 27504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:53.865672 27504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:53.865705 27504 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:43:53.865729 27504 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:43:53.867209 27504 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836772852275-6085-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38619
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836772852275-6085-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38619
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27504
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:43:53.867484 27504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:53.867717 27504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:53.872678 27510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:53.872758 27509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:53.872694 27512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:53.872752 27504 server_base.cc:1034] running on GCE node
I20260619 02:43:53.873083 27504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:53.873330 27504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:53.874493 27504 hybrid_clock.cc:648] HybridClock initialized: now 1781837033874475 us; error 33 us; skew 500 ppm
Jun 19 02:43:53 dist-test-slave-zpbg krb5kdc[27491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837033, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:53.877122 27504 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:43:53.877561 27504 webserver.cc:460] Webserver started at http://127.0.0.1:34955/ using document root <none> and password file <none>
I20260619 02:43:53.877803 27504 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:53.877849 27504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:53.878021 27504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:43:53.879179 27504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836772852275-6085-0/minicluster-data/master-0/data/instance:
uuid: "4659a7d6ccc249508ba7edcd65957fb8"
format_stamp: "Formatted at 2026-06-19 02:43:53 on dist-test-slave-zpbg"
server_key: "20acbaf139e5efe020da4f1334a273a4"
server_key_iv: "95f8803396cf08bd96d3b3e72912fcf4"
server_key_version: "encryptionkey@0"
I20260619 02:43:53.879520 27504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836772852275-6085-0/minicluster-data/master-0/wal/instance:
uuid: "4659a7d6ccc249508ba7edcd65957fb8"
format_stamp: "Formatted at 2026-06-19 02:43:53 on dist-test-slave-zpbg"
server_key: "20acbaf139e5efe020da4f1334a273a4"
server_key_iv: "95f8803396cf08bd96d3b3e72912fcf4"
server_key_version: "encryptionkey@0"
I20260619 02:43:53.880937 27504 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:43:53.881767 27519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:53.882021 27504 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:43:53.882140 27504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "4659a7d6ccc249508ba7edcd65957fb8"
format_stamp: "Formatted at 2026-06-19 02:43:53 on dist-test-slave-zpbg"
server_key: "20acbaf139e5efe020da4f1334a273a4"
server_key_iv: "95f8803396cf08bd96d3b3e72912fcf4"
server_key_version: "encryptionkey@0"
I20260619 02:43:53.882241 27504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:53.885620 27504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:53.887941 27504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:53.888103 27504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:53.891961 27504 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38619
I20260619 02:43:53.891993 27571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38619 every 8 connection(s)
I20260619 02:43:53.892318 27504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:43:53.893244 27572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:53.895448 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 27504
I20260619 02:43:53.895565 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836772852275-6085-0/minicluster-data/master-0/wal/instance
I20260619 02:43:53.895691 27572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4659a7d6ccc249508ba7edcd65957fb8: Bootstrap starting.
I20260619 02:43:53.895735 6085 external_mini_cluster.cc:1428] Setting key 0a8690db13cfc5ca0af065391e88598e
I20260619 02:43:53.896381 27572 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4659a7d6ccc249508ba7edcd65957fb8: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:53.896703 27572 log.cc:824] T 00000000000000000000000000000000 P 4659a7d6ccc249508ba7edcd65957fb8: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:53.897478 27572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4659a7d6ccc249508ba7edcd65957fb8: No bootstrap required, opened a new log
I20260619 02:43:53.898356 27575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:53.896212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.226:56035 (local address 127.0.0.1:38619)
0619 02:43:53.896405 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:53.896412 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:53.896461 (+ 49us) server_negotiation.cc:407] Connection header received
0619 02:43:53.897037 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:53.897046 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:53.897081 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:53.897221 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:53.897895 (+ 674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:53.897898 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:53.897898 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:43:53.897902 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:53.897939 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:43:53.897949 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:53.898034 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:53.898092 (+ 58us) server_negotiation.cc:299] Negotiation successful
0619 02:43:53.898125 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":92,"threads_started":1}
I20260619 02:43:53.899562 6085 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:43:53.900064 27572 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4659a7d6ccc249508ba7edcd65957fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4659a7d6ccc249508ba7edcd65957fb8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38619 } }
I20260619 02:43:53.900246 27572 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4659a7d6ccc249508ba7edcd65957fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:53.900279 27572 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4659a7d6ccc249508ba7edcd65957fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4659a7d6ccc249508ba7edcd65957fb8, State: Initialized, Role: FOLLOWER
I20260619 02:43:53.900403 27572 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4659a7d6ccc249508ba7edcd65957fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4659a7d6ccc249508ba7edcd65957fb8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38619 } }
I20260619 02:43:53.900489 27572 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4659a7d6ccc249508ba7edcd65957fb8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:43:53.900528 27572 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4659a7d6ccc249508ba7edcd65957fb8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:43:53.900576 27572 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4659a7d6ccc249508ba7edcd65957fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:53.901358 27572 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4659a7d6ccc249508ba7edcd65957fb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4659a7d6ccc249508ba7edcd65957fb8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38619 } }
I20260619 02:43:53.901474 27572 leader_election.cc:302] T 00000000000000000000000000000000 P 4659a7d6ccc249508ba7edcd65957fb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4659a7d6ccc249508ba7edcd65957fb8; no voters:
I20260619 02:43:53.901641 27572 leader_election.cc:288] T 00000000000000000000000000000000 P 4659a7d6ccc249508ba7edcd65957fb8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:43:53.901682 27582 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4659a7d6ccc249508ba7edcd65957fb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:43:53.901826 27582 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4659a7d6ccc249508ba7edcd65957fb8 [term 1 LEADER]: Becoming Leader. State: Replica: 4659a7d6ccc249508ba7edcd65957fb8, State: Running, Role: LEADER
I20260619 02:43:53.901908 27572 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4659a7d6ccc249508ba7edcd65957fb8 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:43:53.901964 27582 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4659a7d6ccc249508ba7edcd65957fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4659a7d6ccc249508ba7edcd65957fb8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38619 } }
I20260619 02:43:53.902402 27583 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4659a7d6ccc249508ba7edcd65957fb8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4659a7d6ccc249508ba7edcd65957fb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4659a7d6ccc249508ba7edcd65957fb8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38619 } } }
I20260619 02:43:53.902624 27575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:53.901141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.226:53603 (local address 127.0.0.1:38619)
0619 02:43:53.901180 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:53.901182 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:53.901335 (+ 153us) server_negotiation.cc:407] Connection header received
0619 02:43:53.901376 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:53.901377 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:53.901382 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:53.901457 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:53.902325 (+ 868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:53.902327 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:53.902327 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:43:53.902331 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:53.902346 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:43:53.902351 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:53.902400 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:53.902543 (+ 143us) server_negotiation.cc:299] Negotiation successful
0619 02:43:53.902558 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:43:53.902742 27583 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4659a7d6ccc249508ba7edcd65957fb8 [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:53.902661 27584 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4659a7d6ccc249508ba7edcd65957fb8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4659a7d6ccc249508ba7edcd65957fb8. Latest consensus state: current_term: 1 leader_uuid: "4659a7d6ccc249508ba7edcd65957fb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4659a7d6ccc249508ba7edcd65957fb8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38619 } } }
I20260619 02:43:53.903261 27584 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4659a7d6ccc249508ba7edcd65957fb8 [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:53.903244 27591 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:43:53.903908 27591 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:43:53.905265 27591 catalog_manager.cc:1347] Generated new cluster ID: 95eb7c3164de4026a8120b117334ba6e
I20260619 02:43:53.905318 27591 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:43:53.917234 27591 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:43:53.917418 27591 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:43:53.926017 27591 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4659a7d6ccc249508ba7edcd65957fb8: Generated new TSK 0
I20260619 02:43:53.926275 27591 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:43:53.931440 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 27504
2026-06-19T02:43:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1299 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836772852275-6085-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:43:53 dist-test-slave-zpbg krb5kdc[27603](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:43:53 dist-test-slave-zpbg krb5kdc[27603](info): set up 2 sockets
Jun 19 02:43:53 dist-test-slave-zpbg krb5kdc[27603](info): commencing operation
krb5kdc: starting...
W20260619 02:43:54.970851 6085 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Jun 19 02:43:55 dist-test-slave-zpbg krb5kdc[27603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837035, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:43:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:43:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.241.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
I20260619 02:43:55.089119 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:45229
--webserver_interface=127.5.241.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:44721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:45229
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:43:55.169369 27616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:55.169572 27616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:55.169595 27616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:55.170763 27616 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:43:55.170827 27616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:55.170841 27616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:55.170853 27616 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:43:55.170866 27616 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:43:55.172348 27616 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:44721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836772852275-6085-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:45229
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:45229
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.241.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:43:55.172598 27616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:55.172837 27616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:55.177830 27621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:55.177837 27622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:55.178066 27616 server_base.cc:1034] running on GCE node
W20260619 02:43:55.177847 27624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:55.178354 27616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:55.178613 27616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:55.179752 27616 hybrid_clock.cc:648] HybridClock initialized: now 1781837035179724 us; error 41 us; skew 500 ppm
Jun 19 02:43:55 dist-test-slave-zpbg krb5kdc[27603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837035, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:55.182171 27616 init.cc:377] Logged in from keytab as kudu/127.5.241.126@KRBTEST.COM (short username kudu)
I20260619 02:43:55.182544 27616 webserver.cc:460] Webserver started at http://127.5.241.126:38389/ using document root <none> and password file <none>
I20260619 02:43:55.182739 27616 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:55.182826 27616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:55.182932 27616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:43:55.183919 27616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836772852275-6085-0/minicluster-data/master-0/data/instance:
uuid: "2dc6c395630a491dbb5f6099a4593fca"
format_stamp: "Formatted at 2026-06-19 02:43:55 on dist-test-slave-zpbg"
server_key: "682a2ada0b896bfb459bf3b916302b9f"
server_key_iv: "c748844c0f3f20214186bc45ae273052"
server_key_version: "encryptionkey@0"
I20260619 02:43:55.184212 27616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836772852275-6085-0/minicluster-data/master-0/wal/instance:
uuid: "2dc6c395630a491dbb5f6099a4593fca"
format_stamp: "Formatted at 2026-06-19 02:43:55 on dist-test-slave-zpbg"
server_key: "682a2ada0b896bfb459bf3b916302b9f"
server_key_iv: "c748844c0f3f20214186bc45ae273052"
server_key_version: "encryptionkey@0"
I20260619 02:43:55.185505 27616 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 02:43:55.186233 27631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:55.186444 27616 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:43:55.186555 27616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "2dc6c395630a491dbb5f6099a4593fca"
format_stamp: "Formatted at 2026-06-19 02:43:55 on dist-test-slave-zpbg"
server_key: "682a2ada0b896bfb459bf3b916302b9f"
server_key_iv: "c748844c0f3f20214186bc45ae273052"
server_key_version: "encryptionkey@0"
I20260619 02:43:55.186636 27616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:55.196926 27616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:55.198674 27616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:55.198901 27616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:55.202569 27616 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.126:45229
I20260619 02:43:55.202605 27683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.126:45229 every 8 connection(s)
I20260619 02:43:55.203002 27616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:43:55.203727 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 27616
I20260619 02:43:55.203845 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836772852275-6085-0/minicluster-data/master-0/wal/instance
I20260619 02:43:55.203962 27684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:55.204043 6085 external_mini_cluster.cc:1428] Setting key 420000f021a341d16fb1d9933c1a01b5
I20260619 02:43:55.207340 27684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2dc6c395630a491dbb5f6099a4593fca: Bootstrap starting.
I20260619 02:43:55.208099 27684 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2dc6c395630a491dbb5f6099a4593fca: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:55.208389 27684 log.cc:824] T 00000000000000000000000000000000 P 2dc6c395630a491dbb5f6099a4593fca: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:55.209327 27684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2dc6c395630a491dbb5f6099a4593fca: No bootstrap required, opened a new log
Jun 19 02:43:55 dist-test-slave-zpbg krb5kdc[27603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837035, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:43:55.211308 27684 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2dc6c395630a491dbb5f6099a4593fca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dc6c395630a491dbb5f6099a4593fca" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45229 } }
I20260619 02:43:55.211509 27684 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2dc6c395630a491dbb5f6099a4593fca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:55.211566 27684 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2dc6c395630a491dbb5f6099a4593fca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2dc6c395630a491dbb5f6099a4593fca, State: Initialized, Role: FOLLOWER
I20260619 02:43:55.211727 27684 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2dc6c395630a491dbb5f6099a4593fca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dc6c395630a491dbb5f6099a4593fca" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45229 } }
I20260619 02:43:55.211822 27684 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2dc6c395630a491dbb5f6099a4593fca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:43:55.211856 27684 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2dc6c395630a491dbb5f6099a4593fca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:43:55.211954 27684 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2dc6c395630a491dbb5f6099a4593fca [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:55.212484 27684 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2dc6c395630a491dbb5f6099a4593fca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dc6c395630a491dbb5f6099a4593fca" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45229 } }
I20260619 02:43:55.212604 27684 leader_election.cc:302] T 00000000000000000000000000000000 P 2dc6c395630a491dbb5f6099a4593fca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2dc6c395630a491dbb5f6099a4593fca; no voters:
I20260619 02:43:55.212809 27684 leader_election.cc:288] T 00000000000000000000000000000000 P 2dc6c395630a491dbb5f6099a4593fca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:43:55.212841 27689 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2dc6c395630a491dbb5f6099a4593fca [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:43:55.212984 27689 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2dc6c395630a491dbb5f6099a4593fca [term 1 LEADER]: Becoming Leader. State: Replica: 2dc6c395630a491dbb5f6099a4593fca, State: Running, Role: LEADER
I20260619 02:43:55.213125 27684 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2dc6c395630a491dbb5f6099a4593fca [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:43:55.213119 27689 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2dc6c395630a491dbb5f6099a4593fca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dc6c395630a491dbb5f6099a4593fca" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45229 } }
I20260619 02:43:55.213586 27690 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dc6c395630a491dbb5f6099a4593fca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2dc6c395630a491dbb5f6099a4593fca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dc6c395630a491dbb5f6099a4593fca" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45229 } } }
I20260619 02:43:55.213847 27690 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dc6c395630a491dbb5f6099a4593fca [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:55.213757 27691 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dc6c395630a491dbb5f6099a4593fca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2dc6c395630a491dbb5f6099a4593fca. Latest consensus state: current_term: 1 leader_uuid: "2dc6c395630a491dbb5f6099a4593fca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dc6c395630a491dbb5f6099a4593fca" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45229 } } }
I20260619 02:43:55.214231 27691 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dc6c395630a491dbb5f6099a4593fca [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:55.214828 27687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:55.204523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43067 (local address 127.5.241.126:45229)
0619 02:43:55.204697 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:55.204703 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:55.204728 (+ 25us) server_negotiation.cc:407] Connection header received
0619 02:43:55.205119 (+ 391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:55.205125 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:55.205168 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:55.205279 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:55.206075 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:55.206955 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:55.207989 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:55.208443 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:55.208567 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:55.210933 (+ 2366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:55.210937 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:55.210937 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:55.210947 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:55.212725 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:55.213182 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:55.213185 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:55.213185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:55.213279 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:55.213502 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:55.213503 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:55.213504 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:55.213848 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:55.214012 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:55.214502 (+ 490us) server_negotiation.cc:299] Negotiation successful
0619 02:43:55.214600 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":92,"threads_started":1}
W20260619 02:43:55.215121 27705 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2dc6c395630a491dbb5f6099a4593fca: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:43:55.215168 27705 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:43:55.215263 27699 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:43:55.215406 27699 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:43:55.216008 6085 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:43:55.217298 27699 catalog_manager.cc:1347] Generated new cluster ID: f14c61f0afb04ccbb9157e3d22ddd811
I20260619 02:43:55.217397 27699 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:43:55.223464 27687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:55.217406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39993 (local address 127.5.241.126:45229)
0619 02:43:55.217439 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:55.217441 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:55.217598 (+ 157us) server_negotiation.cc:407] Connection header received
0619 02:43:55.217669 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:55.217670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:55.217680 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:55.217727 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:55.218214 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:55.218692 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:55.219420 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:55.219646 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:55.219720 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:55.220835 (+ 1115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:55.220836 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:55.220837 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:55.220843 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:55.222282 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:55.222656 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:55.222657 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:55.222658 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:55.222716 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:55.222959 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:55.222960 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:55.222960 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:55.223069 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:55.223182 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:55.223309 (+ 127us) server_negotiation.cc:299] Negotiation successful
0619 02:43:55.223371 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260619 02:43:55.236577 27699 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:43:55.237156 27699 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:43:55.240152 27699 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2dc6c395630a491dbb5f6099a4593fca: Generated new TSK 0
I20260619 02:43:55.240372 27699 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:43:55.243394 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 27616
W20260619 02:43:55.247627 27601 connection.cc:535] client connection to 127.5.241.126:45229 recv error: Network error: failed to read from TLS socket (remote: 127.5.241.126:45229): Connection reset by peer (error 104)
2026-06-19T02:43:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1311 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836772852275-6085-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:43:55 dist-test-slave-zpbg krb5kdc[27715](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:43:55 dist-test-slave-zpbg krb5kdc[27715](info): set up 2 sockets
Jun 19 02:43:55 dist-test-slave-zpbg krb5kdc[27715](info): commencing operation
krb5kdc: starting...
W20260619 02:43:56.282951 6085 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Jun 19 02:43:56 dist-test-slave-zpbg krb5kdc[27715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:43:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:43:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.241.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
I20260619 02:43:56.403599 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:37573
--webserver_interface=127.5.241.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:38269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:37573
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:43:56.485415 27728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:56.485636 27728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:56.485661 27728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:56.486860 27728 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:43:56.486904 27728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:56.486919 27728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:56.486933 27728 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:43:56.486945 27728 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:43:56.488449 27728 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:38269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836772852275-6085-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:37573
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:37573
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.241.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27728
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:43:56.488721 27728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:56.488960 27728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:56.493899 27736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:56.493896 27734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:56.493916 27733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:56.494174 27728 server_base.cc:1034] running on GCE node
I20260619 02:43:56.494427 27728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:56.494735 27728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:56.495926 27728 hybrid_clock.cc:648] HybridClock initialized: now 1781837036495904 us; error 38 us; skew 500 ppm
Jun 19 02:43:56 dist-test-slave-zpbg krb5kdc[27715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837036, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:56.498488 27728 init.cc:377] Logged in from keytab as kudu/127.5.241.126@KRBTEST.COM (short username kudu)
I20260619 02:43:56.499020 27728 webserver.cc:460] Webserver started at http://127.5.241.126:38273/ using document root <none> and password file <none>
I20260619 02:43:56.499257 27728 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:56.499325 27728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:56.499475 27728 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:43:56.500546 27728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836772852275-6085-0/minicluster-data/master-0/data/instance:
uuid: "fe0f2a034fcf4f0eb96ea4a167c270e4"
format_stamp: "Formatted at 2026-06-19 02:43:56 on dist-test-slave-zpbg"
server_key: "1fed9abe2a466dfe857a49209e6c1a64"
server_key_iv: "cb9c8e78fb66cfdc696be8c1fa1b0909"
server_key_version: "encryptionkey@0"
I20260619 02:43:56.500931 27728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836772852275-6085-0/minicluster-data/master-0/wal/instance:
uuid: "fe0f2a034fcf4f0eb96ea4a167c270e4"
format_stamp: "Formatted at 2026-06-19 02:43:56 on dist-test-slave-zpbg"
server_key: "1fed9abe2a466dfe857a49209e6c1a64"
server_key_iv: "cb9c8e78fb66cfdc696be8c1fa1b0909"
server_key_version: "encryptionkey@0"
I20260619 02:43:56.502301 27728 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 02:43:56.503228 27743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:56.503437 27728 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:43:56.503543 27728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "fe0f2a034fcf4f0eb96ea4a167c270e4"
format_stamp: "Formatted at 2026-06-19 02:43:56 on dist-test-slave-zpbg"
server_key: "1fed9abe2a466dfe857a49209e6c1a64"
server_key_iv: "cb9c8e78fb66cfdc696be8c1fa1b0909"
server_key_version: "encryptionkey@0"
I20260619 02:43:56.503618 27728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:56.517194 27728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:56.519328 27728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:56.519495 27728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:56.523715 27728 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.126:37573
I20260619 02:43:56.523772 27795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.126:37573 every 8 connection(s)
I20260619 02:43:56.524132 27728 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:43:56.524966 27796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:56.527235 27796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe0f2a034fcf4f0eb96ea4a167c270e4: Bootstrap starting.
I20260619 02:43:56.527813 27796 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fe0f2a034fcf4f0eb96ea4a167c270e4: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:56.528091 27796 log.cc:824] T 00000000000000000000000000000000 P fe0f2a034fcf4f0eb96ea4a167c270e4: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:56.528407 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 27728
I20260619 02:43:56.528533 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836772852275-6085-0/minicluster-data/master-0/wal/instance
I20260619 02:43:56.528715 6085 external_mini_cluster.cc:1428] Setting key 35c7b094006c47d4af50630ab446304e
I20260619 02:43:56.528934 27796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe0f2a034fcf4f0eb96ea4a167c270e4: No bootstrap required, opened a new log
I20260619 02:43:56.530704 27796 raft_consensus.cc:348] T 00000000000000000000000000000000 P fe0f2a034fcf4f0eb96ea4a167c270e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe0f2a034fcf4f0eb96ea4a167c270e4" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 37573 } }
I20260619 02:43:56.530931 27796 raft_consensus.cc:374] T 00000000000000000000000000000000 P fe0f2a034fcf4f0eb96ea4a167c270e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:56.531009 27796 raft_consensus.cc:729] T 00000000000000000000000000000000 P fe0f2a034fcf4f0eb96ea4a167c270e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe0f2a034fcf4f0eb96ea4a167c270e4, State: Initialized, Role: FOLLOWER
I20260619 02:43:56.531155 27796 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe0f2a034fcf4f0eb96ea4a167c270e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe0f2a034fcf4f0eb96ea4a167c270e4" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 37573 } }
I20260619 02:43:56.531250 27796 raft_consensus.cc:388] T 00000000000000000000000000000000 P fe0f2a034fcf4f0eb96ea4a167c270e4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:43:56.531302 27796 raft_consensus.cc:482] T 00000000000000000000000000000000 P fe0f2a034fcf4f0eb96ea4a167c270e4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:43:56.531351 27796 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fe0f2a034fcf4f0eb96ea4a167c270e4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:56.531894 27796 raft_consensus.cc:504] T 00000000000000000000000000000000 P fe0f2a034fcf4f0eb96ea4a167c270e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe0f2a034fcf4f0eb96ea4a167c270e4" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 37573 } }
I20260619 02:43:56.531980 27796 leader_election.cc:302] T 00000000000000000000000000000000 P fe0f2a034fcf4f0eb96ea4a167c270e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fe0f2a034fcf4f0eb96ea4a167c270e4; no voters:
I20260619 02:43:56.532158 27796 leader_election.cc:288] T 00000000000000000000000000000000 P fe0f2a034fcf4f0eb96ea4a167c270e4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:43:56.532196 27801 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fe0f2a034fcf4f0eb96ea4a167c270e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:43:56.532313 27801 raft_consensus.cc:686] T 00000000000000000000000000000000 P fe0f2a034fcf4f0eb96ea4a167c270e4 [term 1 LEADER]: Becoming Leader. State: Replica: fe0f2a034fcf4f0eb96ea4a167c270e4, State: Running, Role: LEADER
I20260619 02:43:56.532428 27796 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe0f2a034fcf4f0eb96ea4a167c270e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:43:56.532442 27801 consensus_queue.cc:237] T 00000000000000000000000000000000 P fe0f2a034fcf4f0eb96ea4a167c270e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe0f2a034fcf4f0eb96ea4a167c270e4" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 37573 } }
I20260619 02:43:56.532829 27802 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe0f2a034fcf4f0eb96ea4a167c270e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fe0f2a034fcf4f0eb96ea4a167c270e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe0f2a034fcf4f0eb96ea4a167c270e4" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 37573 } } }
I20260619 02:43:56.532846 27803 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe0f2a034fcf4f0eb96ea4a167c270e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fe0f2a034fcf4f0eb96ea4a167c270e4. Latest consensus state: current_term: 1 leader_uuid: "fe0f2a034fcf4f0eb96ea4a167c270e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe0f2a034fcf4f0eb96ea4a167c270e4" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 37573 } } }
I20260619 02:43:56.532922 27802 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe0f2a034fcf4f0eb96ea4a167c270e4 [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:56.532956 27803 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe0f2a034fcf4f0eb96ea4a167c270e4 [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:56.533133 27806 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:43:56.533598 27806 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:43:56.535892 27800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:56.529250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42387 (local address 127.5.241.126:37573)
0619 02:43:56.529425 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:56.529434 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:56.529469 (+ 35us) server_negotiation.cc:407] Connection header received
0619 02:43:56.530703 (+ 1234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:56.530710 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:56.530762 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:56.530943 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:56.531181 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:56.532638 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:56.533766 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:56.534350 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:56.534479 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:56.535228 (+ 749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:56.535231 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:56.535232 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:43:56.535404 (+ 172us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:56.535434 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:43:56.535444 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:56.535534 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:56.535615 (+ 81us) server_negotiation.cc:299] Negotiation successful
0619 02:43:56.535645 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":96,"threads_started":1}
I20260619 02:43:56.536263 27806 catalog_manager.cc:1347] Generated new cluster ID: 2d2e4fcfa1ef465cbd9a3f3e26824d81
I20260619 02:43:56.536324 27806 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:43:56.536872 6085 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:43:56.541795 27800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:56.538289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40421 (local address 127.5.241.126:37573)
0619 02:43:56.538323 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:56.538325 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:56.538410 (+ 85us) server_negotiation.cc:407] Connection header received
0619 02:43:56.538469 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:56.538470 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:56.538480 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:56.538536 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:56.538649 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:56.539357 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:56.540442 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:56.540775 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:56.540833 (+ 58us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:56.541544 (+ 711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:56.541545 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:56.541546 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:43:56.541550 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:56.541564 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:43:56.541568 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:56.541617 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:56.541710 (+ 93us) server_negotiation.cc:299] Negotiation successful
0619 02:43:56.541722 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260619 02:43:56.552409 27806 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:43:56.553148 27806 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:43:56.563407 27806 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fe0f2a034fcf4f0eb96ea4a167c270e4: Generated new TSK 0
I20260619 02:43:56.563607 27806 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:43:56.569655 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 27728
W20260619 02:43:56.573838 27713 connection.cc:535] client connection to 127.5.241.126:37573 recv error: Network error: failed to read from TLS socket (remote: 127.5.241.126:37573): Connection reset by peer (error 104)
2026-06-19T02:43:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1326 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836772852275-6085-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:43:56 dist-test-slave-zpbg krb5kdc[27827](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:43:56 dist-test-slave-zpbg krb5kdc[27827](info): set up 2 sockets
Jun 19 02:43:56 dist-test-slave-zpbg krb5kdc[27827](info): commencing operation
krb5kdc: starting...
W20260619 02:43:57.614892 6085 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Jun 19 02:43:57 dist-test-slave-zpbg krb5kdc[27827](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837037, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:43:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:43:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.241.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
I20260619 02:43:57.731742 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:38465
--webserver_interface=127.5.241.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:39599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:38465
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:43:57.812831 27840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:57.813095 27840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:57.813138 27840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:57.814307 27840 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:43:57.814375 27840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:57.814399 27840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:57.814427 27840 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:43:57.814457 27840 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:43:57.815986 27840 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:39599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836772852275-6085-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:38465
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:38465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.241.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:43:57.816251 27840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:57.816483 27840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:57.821244 27845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:57.821252 27846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:57.821280 27840 server_base.cc:1034] running on GCE node
W20260619 02:43:57.821255 27848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:57.821768 27840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:57.822018 27840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:57.823150 27840 hybrid_clock.cc:648] HybridClock initialized: now 1781837037823137 us; error 33 us; skew 500 ppm
Jun 19 02:43:57 dist-test-slave-zpbg krb5kdc[27827](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837037, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:57.825733 27840 init.cc:377] Logged in from keytab as kudu/127.5.241.126@KRBTEST.COM (short username kudu)
I20260619 02:43:57.826138 27840 webserver.cc:460] Webserver started at http://127.5.241.126:36739/ using document root <none> and password file <none>
I20260619 02:43:57.826380 27840 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:57.826424 27840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:57.826578 27840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:43:57.827646 27840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836772852275-6085-0/minicluster-data/master-0/data/instance:
uuid: "9a84eabf833b4768b08a0a3adf671fca"
format_stamp: "Formatted at 2026-06-19 02:43:57 on dist-test-slave-zpbg"
server_key: "06ffe838a86affe4c36180821a03b312"
server_key_iv: "550f6b310b6655aa83141a7027728721"
server_key_version: "encryptionkey@0"
I20260619 02:43:57.827991 27840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836772852275-6085-0/minicluster-data/master-0/wal/instance:
uuid: "9a84eabf833b4768b08a0a3adf671fca"
format_stamp: "Formatted at 2026-06-19 02:43:57 on dist-test-slave-zpbg"
server_key: "06ffe838a86affe4c36180821a03b312"
server_key_iv: "550f6b310b6655aa83141a7027728721"
server_key_version: "encryptionkey@0"
I20260619 02:43:57.829306 27840 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:43:57.830060 27855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:57.830242 27840 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:43:57.830340 27840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "9a84eabf833b4768b08a0a3adf671fca"
format_stamp: "Formatted at 2026-06-19 02:43:57 on dist-test-slave-zpbg"
server_key: "06ffe838a86affe4c36180821a03b312"
server_key_iv: "550f6b310b6655aa83141a7027728721"
server_key_version: "encryptionkey@0"
I20260619 02:43:57.830420 27840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:57.833899 27840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:57.836076 27840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:57.836234 27840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:57.840010 27840 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.126:38465
I20260619 02:43:57.840044 27907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.126:38465 every 8 connection(s)
I20260619 02:43:57.840371 27840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:43:57.841336 27908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:57.843676 27908 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a84eabf833b4768b08a0a3adf671fca: Bootstrap starting.
I20260619 02:43:57.844246 27908 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a84eabf833b4768b08a0a3adf671fca: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:57.844528 27908 log.cc:824] T 00000000000000000000000000000000 P 9a84eabf833b4768b08a0a3adf671fca: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:57.845211 27908 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a84eabf833b4768b08a0a3adf671fca: No bootstrap required, opened a new log
I20260619 02:43:57.846303 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 27840
I20260619 02:43:57.846422 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836772852275-6085-0/minicluster-data/master-0/wal/instance
I20260619 02:43:57.846607 6085 external_mini_cluster.cc:1428] Setting key 2cd5c2128240d5cee94baaa830299938
I20260619 02:43:57.847241 27908 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a84eabf833b4768b08a0a3adf671fca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a84eabf833b4768b08a0a3adf671fca" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 38465 } }
I20260619 02:43:57.847436 27908 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9a84eabf833b4768b08a0a3adf671fca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:57.847486 27908 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a84eabf833b4768b08a0a3adf671fca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a84eabf833b4768b08a0a3adf671fca, State: Initialized, Role: FOLLOWER
I20260619 02:43:57.847631 27908 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a84eabf833b4768b08a0a3adf671fca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a84eabf833b4768b08a0a3adf671fca" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 38465 } }
I20260619 02:43:57.847718 27908 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a84eabf833b4768b08a0a3adf671fca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:43:57.847779 27908 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a84eabf833b4768b08a0a3adf671fca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:43:57.847851 27908 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a84eabf833b4768b08a0a3adf671fca [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:57.848779 27912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:57.847168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58813 (local address 127.5.241.126:38465)
0619 02:43:57.847387 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:57.847395 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:57.847419 (+ 24us) server_negotiation.cc:407] Connection header received
0619 02:43:57.847545 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:57.847554 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:57.847584 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:57.847725 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:57.848376 (+ 651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:57.848379 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:57.848379 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:43:57.848383 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:57.848430 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:43:57.848440 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:57.848511 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:57.848581 (+ 70us) server_negotiation.cc:299] Negotiation successful
0619 02:43:57.848624 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":106,"threads_started":1}
I20260619 02:43:57.848934 27908 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a84eabf833b4768b08a0a3adf671fca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a84eabf833b4768b08a0a3adf671fca" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 38465 } }
I20260619 02:43:57.849042 27908 leader_election.cc:302] T 00000000000000000000000000000000 P 9a84eabf833b4768b08a0a3adf671fca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a84eabf833b4768b08a0a3adf671fca; no voters:
I20260619 02:43:57.849207 27908 leader_election.cc:288] T 00000000000000000000000000000000 P 9a84eabf833b4768b08a0a3adf671fca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:43:57.849254 27913 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a84eabf833b4768b08a0a3adf671fca [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:43:57.849395 27913 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a84eabf833b4768b08a0a3adf671fca [term 1 LEADER]: Becoming Leader. State: Replica: 9a84eabf833b4768b08a0a3adf671fca, State: Running, Role: LEADER
I20260619 02:43:57.849440 27908 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a84eabf833b4768b08a0a3adf671fca [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:43:57.849545 27913 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a84eabf833b4768b08a0a3adf671fca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a84eabf833b4768b08a0a3adf671fca" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 38465 } }
I20260619 02:43:57.849697 6085 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:43:57.850632 27914 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a84eabf833b4768b08a0a3adf671fca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a84eabf833b4768b08a0a3adf671fca. Latest consensus state: current_term: 1 leader_uuid: "9a84eabf833b4768b08a0a3adf671fca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a84eabf833b4768b08a0a3adf671fca" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 38465 } } }
I20260619 02:43:57.850749 27914 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a84eabf833b4768b08a0a3adf671fca [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:57.850970 27930 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:43:57.851365 27915 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a84eabf833b4768b08a0a3adf671fca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9a84eabf833b4768b08a0a3adf671fca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a84eabf833b4768b08a0a3adf671fca" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 38465 } } }
I20260619 02:43:57.851481 27915 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a84eabf833b4768b08a0a3adf671fca [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:57.851799 27930 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:43:57.852638 27912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:57.851254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47285 (local address 127.5.241.126:38465)
0619 02:43:57.851289 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:57.851291 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:57.851551 (+ 260us) server_negotiation.cc:407] Connection header received
0619 02:43:57.851660 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:57.851661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:57.851666 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:57.851727 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:57.852418 (+ 691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:57.852419 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:57.852420 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:43:57.852424 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:57.852439 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:43:57.852443 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:57.852504 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:57.852563 (+ 59us) server_negotiation.cc:299] Negotiation successful
0619 02:43:57.852576 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 02:43:57.853442 27930 catalog_manager.cc:1347] Generated new cluster ID: 69b6fa9bd57d492fb2e6c9aad304e5d2
I20260619 02:43:57.853504 27930 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:43:57.859814 27930 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:43:57.859978 27930 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:43:57.876508 27930 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a84eabf833b4768b08a0a3adf671fca: Generated new TSK 0
I20260619 02:43:57.876729 27930 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:43:57.884328 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 27840
2026-06-19T02:43:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1314 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836772852275-6085-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:43:57 dist-test-slave-zpbg krb5kdc[27939](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:43:57 dist-test-slave-zpbg krb5kdc[27939](info): set up 2 sockets
Jun 19 02:43:57 dist-test-slave-zpbg krb5kdc[27939](info): commencing operation
krb5kdc: starting...
W20260619 02:43:58.922946 6085 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Jun 19 02:43:58 dist-test-slave-zpbg krb5kdc[27939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:43:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:43:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.241.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
I20260619 02:43:59.041682 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:35113
--webserver_interface=127.5.241.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:46039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:35113
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:43:59.123589 27952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:59.123797 27952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:59.123821 27952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:59.125018 27952 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:43:59.125061 27952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:59.125074 27952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:59.125087 27952 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:43:59.125120 27952 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:43:59.126634 27952 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:46039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836772852275-6085-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:35113
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:35113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.241.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27952
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:43:59.126907 27952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:59.127128 27952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:59.131882 27957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:59.131910 27960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:59.131914 27958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:59.131989 27952 server_base.cc:1034] running on GCE node
I20260619 02:43:59.132334 27952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:59.132645 27952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:59.133809 27952 hybrid_clock.cc:648] HybridClock initialized: now 1781837039133794 us; error 41 us; skew 500 ppm
Jun 19 02:43:59 dist-test-slave-zpbg krb5kdc[27939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837039, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:59.140614 27952 init.cc:377] Logged in from keytab as kudu/127.5.241.126@KRBTEST.COM (short username kudu)
I20260619 02:43:59.141104 27952 webserver.cc:460] Webserver started at http://127.5.241.126:41421/ using document root <none> and password file <none>
I20260619 02:43:59.141326 27952 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:59.141391 27952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:59.141535 27952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:43:59.142621 27952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836772852275-6085-0/minicluster-data/master-0/data/instance:
uuid: "8ddabe055dab410da3b2951b46760221"
format_stamp: "Formatted at 2026-06-19 02:43:59 on dist-test-slave-zpbg"
server_key: "7fc7c238914fb0549f5ee4d4049a830e"
server_key_iv: "b2819a2037abc14590597722963ac075"
server_key_version: "encryptionkey@0"
I20260619 02:43:59.143011 27952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836772852275-6085-0/minicluster-data/master-0/wal/instance:
uuid: "8ddabe055dab410da3b2951b46760221"
format_stamp: "Formatted at 2026-06-19 02:43:59 on dist-test-slave-zpbg"
server_key: "7fc7c238914fb0549f5ee4d4049a830e"
server_key_iv: "b2819a2037abc14590597722963ac075"
server_key_version: "encryptionkey@0"
I20260619 02:43:59.144340 27952 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:43:59.145056 27967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:59.145217 27952 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:43:59.145314 27952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "8ddabe055dab410da3b2951b46760221"
format_stamp: "Formatted at 2026-06-19 02:43:59 on dist-test-slave-zpbg"
server_key: "7fc7c238914fb0549f5ee4d4049a830e"
server_key_iv: "b2819a2037abc14590597722963ac075"
server_key_version: "encryptionkey@0"
I20260619 02:43:59.145390 27952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:59.162276 27952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:59.164237 27952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:59.164393 27952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:59.168376 27952 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.126:35113
I20260619 02:43:59.168411 28019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.126:35113 every 8 connection(s)
I20260619 02:43:59.168793 27952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:43:59.169589 28020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:59.171860 28020 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ddabe055dab410da3b2951b46760221: Bootstrap starting.
I20260619 02:43:59.172364 28020 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ddabe055dab410da3b2951b46760221: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:59.172592 28020 log.cc:824] T 00000000000000000000000000000000 P 8ddabe055dab410da3b2951b46760221: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:59.173328 28020 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ddabe055dab410da3b2951b46760221: No bootstrap required, opened a new log
I20260619 02:43:59.174923 28020 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ddabe055dab410da3b2951b46760221 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ddabe055dab410da3b2951b46760221" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 35113 } }
I20260619 02:43:59.175060 28020 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ddabe055dab410da3b2951b46760221 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:59.175081 28020 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ddabe055dab410da3b2951b46760221 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ddabe055dab410da3b2951b46760221, State: Initialized, Role: FOLLOWER
I20260619 02:43:59.175180 28020 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ddabe055dab410da3b2951b46760221 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ddabe055dab410da3b2951b46760221" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 35113 } }
I20260619 02:43:59.175254 28020 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ddabe055dab410da3b2951b46760221 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:43:59.175292 28020 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ddabe055dab410da3b2951b46760221 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:43:59.175359 28020 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ddabe055dab410da3b2951b46760221 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:59.176173 28020 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ddabe055dab410da3b2951b46760221 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ddabe055dab410da3b2951b46760221" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 35113 } }
I20260619 02:43:59.176290 28020 leader_election.cc:302] T 00000000000000000000000000000000 P 8ddabe055dab410da3b2951b46760221 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ddabe055dab410da3b2951b46760221; no voters:
I20260619 02:43:59.176448 28020 leader_election.cc:288] T 00000000000000000000000000000000 P 8ddabe055dab410da3b2951b46760221 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:43:59.176493 28023 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ddabe055dab410da3b2951b46760221 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:43:59.176592 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 27952
I20260619 02:43:59.176676 28023 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ddabe055dab410da3b2951b46760221 [term 1 LEADER]: Becoming Leader. State: Replica: 8ddabe055dab410da3b2951b46760221, State: Running, Role: LEADER
I20260619 02:43:59.176703 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836772852275-6085-0/minicluster-data/master-0/wal/instance
I20260619 02:43:59.176724 28020 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ddabe055dab410da3b2951b46760221 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:43:59.176769 28023 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ddabe055dab410da3b2951b46760221 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ddabe055dab410da3b2951b46760221" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 35113 } }
I20260619 02:43:59.176883 6085 external_mini_cluster.cc:1428] Setting key 55ede812bb659a7eb574cefe2eb0a924
I20260619 02:43:59.177203 28024 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ddabe055dab410da3b2951b46760221 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8ddabe055dab410da3b2951b46760221" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ddabe055dab410da3b2951b46760221" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 35113 } } }
I20260619 02:43:59.177294 28024 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ddabe055dab410da3b2951b46760221 [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:59.177255 28025 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ddabe055dab410da3b2951b46760221 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ddabe055dab410da3b2951b46760221. Latest consensus state: current_term: 1 leader_uuid: "8ddabe055dab410da3b2951b46760221" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ddabe055dab410da3b2951b46760221" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 35113 } } }
I20260619 02:43:59.177341 28025 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ddabe055dab410da3b2951b46760221 [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:59.177647 28033 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:43:59.178371 28033 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:43:59.180102 28033 catalog_manager.cc:1347] Generated new cluster ID: 23eb890415194788b9274c2445adb2c7
I20260619 02:43:59.180166 28033 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:43:59.184715 28033 catalog_manager.cc:1370] Generated new certificate authority record
Jun 19 02:43:59 dist-test-slave-zpbg krb5kdc[27939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:43:59.185195 28033 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:43:59.188967 28034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:59.177490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59033 (local address 127.5.241.126:35113)
0619 02:43:59.177662 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:59.177672 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:59.177901 (+ 229us) server_negotiation.cc:407] Connection header received
0619 02:43:59.178041 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:59.178049 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:59.178107 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:59.178241 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:59.179156 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:59.180484 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:59.181510 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:59.182136 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:59.182268 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:59.185315 (+ 3047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:59.185319 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:59.185320 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:59.185332 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:59.187361 (+ 2029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:59.187748 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:59.187752 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:59.187752 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:59.187867 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:59.188074 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:59.188074 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:59.188075 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:59.188353 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:59.188510 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:59.188581 (+ 71us) server_negotiation.cc:299] Negotiation successful
0619 02:43:59.188746 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":75,"threads_started":1}
I20260619 02:43:59.190061 6085 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:43:59.193280 28033 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8ddabe055dab410da3b2951b46760221: Generated new TSK 0
I20260619 02:43:59.193449 28033 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:43:59.197938 28034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:59.191442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39181 (local address 127.5.241.126:35113)
0619 02:43:59.191477 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:59.191479 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:59.191594 (+ 115us) server_negotiation.cc:407] Connection header received
0619 02:43:59.191642 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:59.191643 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:59.191653 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:59.191715 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:59.192269 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:59.193055 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:59.193943 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:59.194271 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:59.194338 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:59.195464 (+ 1126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:59.195465 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:59.195465 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:59.195471 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:59.196923 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:59.197236 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:59.197237 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:59.197238 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:59.197296 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:59.197468 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:59.197469 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:59.197469 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:59.197592 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:59.197695 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:59.197771 (+ 76us) server_negotiation.cc:299] Negotiation successful
0619 02:43:59.197849 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 02:43:59.200133 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 27952
2026-06-19T02:43:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1315 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836772852275-6085-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:43:59 dist-test-slave-zpbg krb5kdc[28051](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:43:59 dist-test-slave-zpbg krb5kdc[28051](info): set up 2 sockets
Jun 19 02:43:59 dist-test-slave-zpbg krb5kdc[28051](info): commencing operation
krb5kdc: starting...
W20260619 02:44:00.234998 6085 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Jun 19 02:44:00 dist-test-slave-zpbg krb5kdc[28051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837040, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:44:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:44:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.241.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
I20260619 02:44:00.353767 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:36349
--webserver_interface=127.5.241.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:37161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:36349
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:44:00.434487 28064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:44:00.434729 28064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:44:00.434768 28064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:44:00.435933 28064 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:44:00.436002 28064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:44:00.436036 28064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:44:00.436080 28064 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:44:00.436110 28064 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:44:00.437655 28064 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:37161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836772852275-6085-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:36349
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:36349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.241.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28064
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:44:00.437845 28064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:44:00.438015 28064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:44:00.443754 28070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:00.443727 28069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:00.443735 28072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:00.443909 28064 server_base.cc:1034] running on GCE node
I20260619 02:44:00.444239 28064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:44:00.444581 28064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:44:00.445742 28064 hybrid_clock.cc:648] HybridClock initialized: now 1781837040445730 us; error 31 us; skew 500 ppm
Jun 19 02:44:00 dist-test-slave-zpbg krb5kdc[28051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837040, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:44:00.448369 28064 init.cc:377] Logged in from keytab as kudu/127.5.241.126@KRBTEST.COM (short username kudu)
I20260619 02:44:00.448832 28064 webserver.cc:460] Webserver started at http://127.5.241.126:42261/ using document root <none> and password file <none>
I20260619 02:44:00.449107 28064 fs_manager.cc:359] Metadata directory not provided
I20260619 02:44:00.449174 28064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:44:00.449323 28064 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:44:00.450385 28064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836772852275-6085-0/minicluster-data/master-0/data/instance:
uuid: "65ccb2ca72a9466b8bf3822728edf94c"
format_stamp: "Formatted at 2026-06-19 02:44:00 on dist-test-slave-zpbg"
server_key: "4b748c62237068049a2e8dfe47dc3bc8"
server_key_iv: "79b2575ae8a12b83e3fdbe21cb16fbfc"
server_key_version: "encryptionkey@0"
I20260619 02:44:00.450745 28064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836772852275-6085-0/minicluster-data/master-0/wal/instance:
uuid: "65ccb2ca72a9466b8bf3822728edf94c"
format_stamp: "Formatted at 2026-06-19 02:44:00 on dist-test-slave-zpbg"
server_key: "4b748c62237068049a2e8dfe47dc3bc8"
server_key_iv: "79b2575ae8a12b83e3fdbe21cb16fbfc"
server_key_version: "encryptionkey@0"
I20260619 02:44:00.452112 28064 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:44:00.452862 28079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:00.453048 28064 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:44:00.453150 28064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "65ccb2ca72a9466b8bf3822728edf94c"
format_stamp: "Formatted at 2026-06-19 02:44:00 on dist-test-slave-zpbg"
server_key: "4b748c62237068049a2e8dfe47dc3bc8"
server_key_iv: "79b2575ae8a12b83e3fdbe21cb16fbfc"
server_key_version: "encryptionkey@0"
I20260619 02:44:00.453234 28064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:44:00.483069 28064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:44:00.485416 28064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:44:00.485591 28064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:44:00.489967 28064 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.126:36349
I20260619 02:44:00.490005 28131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.126:36349 every 8 connection(s)
I20260619 02:44:00.490355 28064 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:44:00.491231 28132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:44:00.493489 28132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65ccb2ca72a9466b8bf3822728edf94c: Bootstrap starting.
I20260619 02:44:00.494024 28132 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 65ccb2ca72a9466b8bf3822728edf94c: Neither blocks nor log segments found. Creating new log.
I20260619 02:44:00.494259 28132 log.cc:824] T 00000000000000000000000000000000 P 65ccb2ca72a9466b8bf3822728edf94c: Log is configured to *not* fsync() on all Append() calls
I20260619 02:44:00.494925 28132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65ccb2ca72a9466b8bf3822728edf94c: No bootstrap required, opened a new log
I20260619 02:44:00.496454 28132 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65ccb2ca72a9466b8bf3822728edf94c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65ccb2ca72a9466b8bf3822728edf94c" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 36349 } }
I20260619 02:44:00.496598 28132 raft_consensus.cc:374] T 00000000000000000000000000000000 P 65ccb2ca72a9466b8bf3822728edf94c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:44:00.496628 28132 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65ccb2ca72a9466b8bf3822728edf94c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65ccb2ca72a9466b8bf3822728edf94c, State: Initialized, Role: FOLLOWER
I20260619 02:44:00.496732 28132 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65ccb2ca72a9466b8bf3822728edf94c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65ccb2ca72a9466b8bf3822728edf94c" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 36349 } }
I20260619 02:44:00.496821 28132 raft_consensus.cc:388] T 00000000000000000000000000000000 P 65ccb2ca72a9466b8bf3822728edf94c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:44:00.496861 28132 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65ccb2ca72a9466b8bf3822728edf94c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:44:00.496910 28132 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65ccb2ca72a9466b8bf3822728edf94c [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:44:00.497735 28132 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65ccb2ca72a9466b8bf3822728edf94c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65ccb2ca72a9466b8bf3822728edf94c" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 36349 } }
I20260619 02:44:00.497854 28132 leader_election.cc:302] T 00000000000000000000000000000000 P 65ccb2ca72a9466b8bf3822728edf94c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 65ccb2ca72a9466b8bf3822728edf94c; no voters:
I20260619 02:44:00.498023 28132 leader_election.cc:288] T 00000000000000000000000000000000 P 65ccb2ca72a9466b8bf3822728edf94c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:44:00.498078 28135 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65ccb2ca72a9466b8bf3822728edf94c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:44:00.498246 28135 raft_consensus.cc:686] T 00000000000000000000000000000000 P 65ccb2ca72a9466b8bf3822728edf94c [term 1 LEADER]: Becoming Leader. State: Replica: 65ccb2ca72a9466b8bf3822728edf94c, State: Running, Role: LEADER
I20260619 02:44:00.498281 28132 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65ccb2ca72a9466b8bf3822728edf94c [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:44:00.498334 28135 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65ccb2ca72a9466b8bf3822728edf94c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65ccb2ca72a9466b8bf3822728edf94c" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 36349 } }
I20260619 02:44:00.498554 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 28064
I20260619 02:44:00.498675 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836772852275-6085-0/minicluster-data/master-0/wal/instance
I20260619 02:44:00.498881 6085 external_mini_cluster.cc:1428] Setting key 615ea648095a422eb004a7d46df611e2
I20260619 02:44:00.499504 28136 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65ccb2ca72a9466b8bf3822728edf94c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "65ccb2ca72a9466b8bf3822728edf94c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65ccb2ca72a9466b8bf3822728edf94c" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 36349 } } }
I20260619 02:44:00.499629 28136 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65ccb2ca72a9466b8bf3822728edf94c [sys.catalog]: This master's current role is: LEADER
I20260619 02:44:00.499500 28137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65ccb2ca72a9466b8bf3822728edf94c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65ccb2ca72a9466b8bf3822728edf94c. Latest consensus state: current_term: 1 leader_uuid: "65ccb2ca72a9466b8bf3822728edf94c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65ccb2ca72a9466b8bf3822728edf94c" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 36349 } } }
I20260619 02:44:00.499914 28137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65ccb2ca72a9466b8bf3822728edf94c [sys.catalog]: This master's current role is: LEADER
W20260619 02:44:00.500155 28151 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 65ccb2ca72a9466b8bf3822728edf94c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:44:00.500221 28151 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:44:00.500299 28153 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:44:00.500484 28153 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:44:00.501912 28153 catalog_manager.cc:1347] Generated new cluster ID: 730fde35505b47288c6c498a63264edc
I20260619 02:44:00.501978 28153 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:44:00.505282 28152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:00.499609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33669 (local address 127.5.241.126:36349)
0619 02:44:00.499801 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:00.499810 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:00.500008 (+ 198us) server_negotiation.cc:407] Connection header received
0619 02:44:00.500152 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:00.500159 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:00.500218 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:00.500370 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:00.500602 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:00.501986 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:00.503152 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:00.503760 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:00.503877 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:00.504565 (+ 688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:00.504570 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:00.504570 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:44:00.504786 (+ 216us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:00.504826 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:44:00.504842 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:00.504905 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:00.504983 (+ 78us) server_negotiation.cc:299] Negotiation successful
0619 02:44:00.505056 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":74,"threads_started":1}
I20260619 02:44:00.506201 6085 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:44:00.508833 28153 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:44:00.509465 28153 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:44:00.512161 28152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:00.508590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52483 (local address 127.5.241.126:36349)
0619 02:44:00.508637 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:00.508682 (+ 45us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:00.508804 (+ 122us) server_negotiation.cc:407] Connection header received
0619 02:44:00.508865 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:00.508866 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:00.508877 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:00.508936 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:00.509128 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:00.509814 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:00.510849 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:00.511147 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:00.511208 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:00.511911 (+ 703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:00.511912 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:00.511913 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:44:00.511917 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:00.511932 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:44:00.511936 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:00.511985 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:00.512047 (+ 62us) server_negotiation.cc:299] Negotiation successful
0619 02:44:00.512088 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 02:44:00.516111 28153 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 65ccb2ca72a9466b8bf3822728edf94c: Generated new TSK 0
I20260619 02:44:00.516291 28153 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:44:00.522642 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 28064
2026-06-19T02:44:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1322 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836772852275-6085-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:44:00 dist-test-slave-zpbg krb5kdc[28163](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:44:00 dist-test-slave-zpbg krb5kdc[28163](info): set up 2 sockets
Jun 19 02:44:00 dist-test-slave-zpbg krb5kdc[28163](info): commencing operation
krb5kdc: starting...
W20260619 02:44:01.566972 6085 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Jun 19 02:44:01 dist-test-slave-zpbg krb5kdc[28163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837041, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:44:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:44:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.241.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
I20260619 02:44:01.689178 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:38779
--webserver_interface=127.5.241.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:45807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:38779
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:44:01.771541 28176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:44:01.771755 28176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:44:01.771780 28176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:44:01.772938 28176 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:44:01.772981 28176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:44:01.772996 28176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:44:01.773010 28176 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:44:01.773021 28176 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:44:01.774495 28176 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:45807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836772852275-6085-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:38779
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:38779
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.241.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28176
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:44:01.774746 28176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:44:01.775010 28176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:44:01.779799 28182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:01.779855 28181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:01.779884 28184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:01.780357 28176 server_base.cc:1034] running on GCE node
I20260619 02:44:01.780646 28176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:44:01.780929 28176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:44:01.782099 28176 hybrid_clock.cc:648] HybridClock initialized: now 1781837041782085 us; error 31 us; skew 500 ppm
Jun 19 02:44:01 dist-test-slave-zpbg krb5kdc[28163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837041, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:44:01.784722 28176 init.cc:377] Logged in from keytab as kudu/127.5.241.126@KRBTEST.COM (short username kudu)
I20260619 02:44:01.785332 28176 webserver.cc:460] Webserver started at http://127.5.241.126:35829/ using document root <none> and password file <none>
I20260619 02:44:01.785632 28176 fs_manager.cc:359] Metadata directory not provided
I20260619 02:44:01.785713 28176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:44:01.785873 28176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:44:01.787509 28176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836772852275-6085-0/minicluster-data/master-0/data/instance:
uuid: "5b215b11640b49bf8110604148405f93"
format_stamp: "Formatted at 2026-06-19 02:44:01 on dist-test-slave-zpbg"
server_key: "285751fc75a253b2b1c4795ad501a8ae"
server_key_iv: "cd1136edb9c92e041bca2e627fdb5e02"
server_key_version: "encryptionkey@0"
I20260619 02:44:01.787981 28176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836772852275-6085-0/minicluster-data/master-0/wal/instance:
uuid: "5b215b11640b49bf8110604148405f93"
format_stamp: "Formatted at 2026-06-19 02:44:01 on dist-test-slave-zpbg"
server_key: "285751fc75a253b2b1c4795ad501a8ae"
server_key_iv: "cd1136edb9c92e041bca2e627fdb5e02"
server_key_version: "encryptionkey@0"
I20260619 02:44:01.789947 28176 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260619 02:44:01.790944 28191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:01.791165 28176 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:44:01.791282 28176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "5b215b11640b49bf8110604148405f93"
format_stamp: "Formatted at 2026-06-19 02:44:01 on dist-test-slave-zpbg"
server_key: "285751fc75a253b2b1c4795ad501a8ae"
server_key_iv: "cd1136edb9c92e041bca2e627fdb5e02"
server_key_version: "encryptionkey@0"
I20260619 02:44:01.791373 28176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:44:01.796839 28176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:44:01.799048 28176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:44:01.799156 28176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:44:01.802992 28176 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.126:38779
I20260619 02:44:01.803025 28243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.126:38779 every 8 connection(s)
I20260619 02:44:01.803373 28176 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:44:01.803784 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 28176
I20260619 02:44:01.803908 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836772852275-6085-0/minicluster-data/master-0/wal/instance
I20260619 02:44:01.804180 6085 external_mini_cluster.cc:1428] Setting key 027d7bd65f8879989bee5370ff2b8284
I20260619 02:44:01.804591 28244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:44:01.806557 28247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:01.804705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34755 (local address 127.5.241.126:38779)
0619 02:44:01.804868 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:01.804876 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:01.804910 (+ 34us) server_negotiation.cc:407] Connection header received
0619 02:44:01.805346 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:01.805352 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:01.805376 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:01.805493 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:01.806121 (+ 628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:01.806124 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:01.806124 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:44:01.806128 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:01.806162 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:44:01.806171 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:01.806226 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:01.806317 (+ 91us) server_negotiation.cc:299] Negotiation successful
0619 02:44:01.806372 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":86,"threads_started":1}
I20260619 02:44:01.807566 6085 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:44:01.808121 28244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b215b11640b49bf8110604148405f93: Bootstrap starting.
I20260619 02:44:01.808861 28244 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b215b11640b49bf8110604148405f93: Neither blocks nor log segments found. Creating new log.
I20260619 02:44:01.809198 28244 log.cc:824] T 00000000000000000000000000000000 P 5b215b11640b49bf8110604148405f93: Log is configured to *not* fsync() on all Append() calls
I20260619 02:44:01.810052 28244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b215b11640b49bf8110604148405f93: No bootstrap required, opened a new log
I20260619 02:44:01.810223 28247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:01.809172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48653 (local address 127.5.241.126:38779)
0619 02:44:01.809212 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:01.809214 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:01.809284 (+ 70us) server_negotiation.cc:407] Connection header received
0619 02:44:01.809361 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:01.809362 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:01.809370 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:01.809436 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:01.809976 (+ 540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:01.809977 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:01.809977 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:44:01.809980 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:01.809992 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:44:01.809995 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:01.810041 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:01.810137 (+ 96us) server_negotiation.cc:299] Negotiation successful
0619 02:44:01.810150 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 02:44:01.812058 28244 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b215b11640b49bf8110604148405f93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b215b11640b49bf8110604148405f93" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 38779 } }
I20260619 02:44:01.812191 28244 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b215b11640b49bf8110604148405f93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:44:01.812213 28244 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b215b11640b49bf8110604148405f93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b215b11640b49bf8110604148405f93, State: Initialized, Role: FOLLOWER
I20260619 02:44:01.812347 28244 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b215b11640b49bf8110604148405f93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b215b11640b49bf8110604148405f93" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 38779 } }
I20260619 02:44:01.812404 28244 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5b215b11640b49bf8110604148405f93 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:44:01.812455 28244 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b215b11640b49bf8110604148405f93 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:44:01.812502 28244 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b215b11640b49bf8110604148405f93 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:44:01.813004 28244 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b215b11640b49bf8110604148405f93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b215b11640b49bf8110604148405f93" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 38779 } }
I20260619 02:44:01.813088 28244 leader_election.cc:302] T 00000000000000000000000000000000 P 5b215b11640b49bf8110604148405f93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5b215b11640b49bf8110604148405f93; no voters:
I20260619 02:44:01.813262 28244 leader_election.cc:288] T 00000000000000000000000000000000 P 5b215b11640b49bf8110604148405f93 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:44:01.813419 28254 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b215b11640b49bf8110604148405f93 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:44:01.813493 28244 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b215b11640b49bf8110604148405f93 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:44:01.813566 28254 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b215b11640b49bf8110604148405f93 [term 1 LEADER]: Becoming Leader. State: Replica: 5b215b11640b49bf8110604148405f93, State: Running, Role: LEADER
I20260619 02:44:01.813663 28254 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b215b11640b49bf8110604148405f93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b215b11640b49bf8110604148405f93" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 38779 } }
I20260619 02:44:01.814162 28255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b215b11640b49bf8110604148405f93 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5b215b11640b49bf8110604148405f93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b215b11640b49bf8110604148405f93" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 38779 } } }
I20260619 02:44:01.814188 28256 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b215b11640b49bf8110604148405f93 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b215b11640b49bf8110604148405f93. Latest consensus state: current_term: 1 leader_uuid: "5b215b11640b49bf8110604148405f93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b215b11640b49bf8110604148405f93" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 38779 } } }
I20260619 02:44:01.814281 28255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b215b11640b49bf8110604148405f93 [sys.catalog]: This master's current role is: LEADER
I20260619 02:44:01.814289 28256 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b215b11640b49bf8110604148405f93 [sys.catalog]: This master's current role is: LEADER
I20260619 02:44:01.814538 28263 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:44:01.814991 28263 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:44:01.816588 28263 catalog_manager.cc:1347] Generated new cluster ID: ff89f7597a89460fad2321833ab1e936
I20260619 02:44:01.816643 28263 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:44:01.830930 28263 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:44:01.831110 28263 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:44:01.834671 28263 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5b215b11640b49bf8110604148405f93: Generated new TSK 0
I20260619 02:44:01.834867 28263 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:44:01.841136 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 28176
2026-06-19T02:44:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1319 ms)
[----------] 18 tests from AuthTokenIssuingTest (23894 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:44:01 dist-test-slave-zpbg krb5kdc[28275](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:44:01 dist-test-slave-zpbg krb5kdc[28275](info): set up 2 sockets
Jun 19 02:44:01 dist-test-slave-zpbg krb5kdc[28275](info): commencing operation
krb5kdc: starting...
W20260619 02:44:02.870971 6085 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Jun 19 02:44:02 dist-test-slave-zpbg krb5kdc[28275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837042, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:44:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:44:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.241.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
I20260619 02:44:02.992023 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:38479
--webserver_interface=127.5.241.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:35911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:38479,127.5.241.125:35077,127.5.241.124:41559
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:44:03.071532 28288 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 02:44:03.071780 28288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:44:03.071811 28288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:44:03.071856 28288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:44:03.073237 28288 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:44:03.073319 28288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:44:03.073348 28288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:44:03.073380 28288 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:44:03.073405 28288 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:44:03.074995 28288 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:35911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:38479,127.5.241.125:35077,127.5.241.124:41559
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:38479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.241.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:44:03.075268 28288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:44:03.075495 28288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:44:03.080118 28296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:03.080174 28293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:03.080288 28288 server_base.cc:1034] running on GCE node
W20260619 02:44:03.080124 28294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:03.080612 28288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:44:03.080900 28288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:44:03.082079 28288 hybrid_clock.cc:648] HybridClock initialized: now 1781837043082063 us; error 26 us; skew 500 ppm
Jun 19 02:44:03 dist-test-slave-zpbg krb5kdc[28275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837043, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:44:03.084659 28288 init.cc:377] Logged in from keytab as kudu/127.5.241.126@KRBTEST.COM (short username kudu)
I20260619 02:44:03.085078 28288 webserver.cc:460] Webserver started at http://127.5.241.126:34981/ using document root <none> and password file <none>
I20260619 02:44:03.085301 28288 fs_manager.cc:359] Metadata directory not provided
I20260619 02:44:03.085366 28288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:44:03.085512 28288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:44:03.086606 28288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/minicluster-data/master-0/data/instance:
uuid: "910d082c01904599ad1bacbe07a76dce"
format_stamp: "Formatted at 2026-06-19 02:44:03 on dist-test-slave-zpbg"
server_key: "6caf5cf15998ef17c79de669c1b66ab3"
server_key_iv: "820215c0dd065d4524c21f39be9a93d3"
server_key_version: "encryptionkey@0"
I20260619 02:44:03.086988 28288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/minicluster-data/master-0/wal/instance:
uuid: "910d082c01904599ad1bacbe07a76dce"
format_stamp: "Formatted at 2026-06-19 02:44:03 on dist-test-slave-zpbg"
server_key: "6caf5cf15998ef17c79de669c1b66ab3"
server_key_iv: "820215c0dd065d4524c21f39be9a93d3"
server_key_version: "encryptionkey@0"
I20260619 02:44:03.088291 28288 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:44:03.089056 28303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:03.089232 28288 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:44:03.089326 28288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "910d082c01904599ad1bacbe07a76dce"
format_stamp: "Formatted at 2026-06-19 02:44:03 on dist-test-slave-zpbg"
server_key: "6caf5cf15998ef17c79de669c1b66ab3"
server_key_iv: "820215c0dd065d4524c21f39be9a93d3"
server_key_version: "encryptionkey@0"
I20260619 02:44:03.089401 28288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:44:03.103657 28288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:44:03.105978 28288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:44:03.106141 28288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:44:03.109997 28288 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.126:38479
I20260619 02:44:03.110055 28355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.126:38479 every 8 connection(s)
I20260619 02:44:03.110401 28288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:44:03.111385 28356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:44:03.112932 28356 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.241.126" port: 38479 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:44:03.116803 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 28288
I20260619 02:44:03.116958 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/minicluster-data/master-0/wal/instance
I20260619 02:44:03.117149 6085 external_mini_cluster.cc:1428] Setting key 468576db73b2c53dedb7cc43eb9c4099
Jun 19 02:44:03 dist-test-slave-zpbg krb5kdc[28275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837043, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:44:03.123229 28358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:03.113629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:38479 (local address 127.0.0.1:34718)
0619 02:44:03.113792 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:03.113853 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:03.114293 (+ 440us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:03.114580 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:03.114582 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:03.114888 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:44:03.115040 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:03.115044 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:03.116069 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:03.116069 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:03.116727 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:03.116728 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:03.117444 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:03.117444 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:03.117503 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:03.118020 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:44:03.118028 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:44:03.119511 (+ 1483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:44:03.121685 (+ 2174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:03.121692 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:03.121693 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:03.121888 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:03.122174 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:03.122174 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:03.122175 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:03.122261 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:03.122739 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:44:03.122743 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:44:03.122849 (+ 106us) client_negotiation.cc:769] Sending connection context
0619 02:44:03.122926 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 02:44:03.123053 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":84,"threads_started":1}
I20260619 02:44:03.123296 28359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:03.113657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34718 (local address 127.5.241.126:38479)
0619 02:44:03.113841 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:03.113850 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:03.113920 (+ 70us) server_negotiation.cc:407] Connection header received
0619 02:44:03.114444 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:03.114446 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:03.114464 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:03.114559 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:03.115143 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:03.115983 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:03.116816 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:03.117341 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:03.117454 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:03.119622 (+ 2168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:03.119630 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:03.119630 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:03.119637 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:03.121591 (+ 1954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:03.121976 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:03.121979 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:03.121980 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:03.122100 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:03.122325 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:03.122326 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:03.122327 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:03.122668 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:03.122784 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:03.122989 (+ 205us) server_negotiation.cc:299] Negotiation successful
0619 02:44:03.123142 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":141,"thread_start_us":114,"threads_started":1}
I20260619 02:44:03.124135 28356 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.241.125" port: 35077 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:44:03.124466 28358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:03.124356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.125:35077 (local address 127.0.0.1:60370)
0619 02:44:03.124402 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:03.124447 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.125:35077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260619 02:44:03.124619 28304 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.241.125:35077: connect: Connection refused (error 111)
I20260619 02:44:03.125039 28358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:03.124958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.125:35077 (local address 127.0.0.1:60384)
0619 02:44:03.125000 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:03.125023 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.125:35077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260619 02:44:03.125154 28356 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.241.125:35077: Network error: Client connection negotiation failed: client connection to 127.5.241.125:35077: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.5.241.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.125.keytab.
Entry for principal kudu/127.5.241.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.125.keytab.
I20260619 02:44:03.142457 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.125:35077
--webserver_interface=127.5.241.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.125.keytab
--principal=kudu/127.5.241.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:35911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:38479,127.5.241.125:35077,127.5.241.124:41559
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:44:03.174369 28356 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.241.125" port: 35077 } attempt: 1
I20260619 02:44:03.174865 28358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:03.174716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.125:35077 (local address 127.0.0.1:60394)
0619 02:44:03.174798 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:03.174836 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.125:35077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260619 02:44:03.175221 28358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:03.175126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.125:35077 (local address 127.0.0.1:60400)
0619 02:44:03.175172 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:03.175196 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.125:35077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260619 02:44:03.175312 28356 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.241.125:35077: Network error: Client connection negotiation failed: client connection to 127.5.241.125:35077: connect: Connection refused (error 111)
W20260619 02:44:03.223078 28363 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 02:44:03.223306 28363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:44:03.223335 28363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:44:03.223376 28363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:44:03.224500 28363 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:44:03.224563 28363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:44:03.224588 28363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:44:03.224617 28363 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:44:03.224644 28363 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:44:03.226104 28363 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:35911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:38479,127.5.241.125:35077,127.5.241.124:41559
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.125.keytab
--principal=kudu/127.5.241.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.125:35077
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.241.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28363
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:44:03.226369 28363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:44:03.226599 28363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:44:03.231366 28368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:03.231360 28371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:03.231379 28363 server_base.cc:1034] running on GCE node
W20260619 02:44:03.231360 28369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:03.231786 28363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:44:03.232075 28363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:44:03.233238 28363 hybrid_clock.cc:648] HybridClock initialized: now 1781837043233203 us; error 53 us; skew 500 ppm
Jun 19 02:44:03 dist-test-slave-zpbg krb5kdc[28275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837043, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:44:03.235831 28363 init.cc:377] Logged in from keytab as kudu/127.5.241.125@KRBTEST.COM (short username kudu)
I20260619 02:44:03.236269 28363 webserver.cc:460] Webserver started at http://127.5.241.125:42131/ using document root <none> and password file <none>
I20260619 02:44:03.236526 28363 fs_manager.cc:359] Metadata directory not provided
I20260619 02:44:03.236609 28363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:44:03.236765 28363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:44:03.237819 28363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/minicluster-data/master-1/data/instance:
uuid: "52f896763048436482043ee9a516226d"
format_stamp: "Formatted at 2026-06-19 02:44:03 on dist-test-slave-zpbg"
server_key: "ef9becb866530b4a39c74c93ddd43b6b"
server_key_iv: "53324d78f806a6ffb3693994c3513ea1"
server_key_version: "encryptionkey@0"
I20260619 02:44:03.238183 28363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/minicluster-data/master-1/wal/instance:
uuid: "52f896763048436482043ee9a516226d"
format_stamp: "Formatted at 2026-06-19 02:44:03 on dist-test-slave-zpbg"
server_key: "ef9becb866530b4a39c74c93ddd43b6b"
server_key_iv: "53324d78f806a6ffb3693994c3513ea1"
server_key_version: "encryptionkey@0"
I20260619 02:44:03.239595 28363 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:44:03.240357 28378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:03.240533 28363 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:44:03.240625 28363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/minicluster-data/master-1/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/minicluster-data/master-1/wal
uuid: "52f896763048436482043ee9a516226d"
format_stamp: "Formatted at 2026-06-19 02:44:03 on dist-test-slave-zpbg"
server_key: "ef9becb866530b4a39c74c93ddd43b6b"
server_key_iv: "53324d78f806a6ffb3693994c3513ea1"
server_key_version: "encryptionkey@0"
I20260619 02:44:03.240700 28363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:44:03.243456 28356 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.241.125" port: 35077 } attempt: 2
I20260619 02:44:03.243885 28358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:03.243740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.125:35077 (local address 127.0.0.1:60416)
0619 02:44:03.243800 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:03.243842 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.125:35077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 02:44:03.244297 28358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:03.244192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.125:35077 (local address 127.0.0.1:60426)
0619 02:44:03.244244 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:03.244275 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.125:35077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260619 02:44:03.244405 28356 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.241.125:35077: Network error: Client connection negotiation failed: client connection to 127.5.241.125:35077: connect: Connection refused (error 111)
I20260619 02:44:03.264163 28363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:44:03.266080 28363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:44:03.266217 28363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:44:03.270134 28363 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.125:35077
I20260619 02:44:03.270177 28430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.125:35077 every 8 connection(s)
I20260619 02:44:03.270532 28363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/minicluster-data/master-1/data/info.pb
I20260619 02:44:03.271517 28431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:44:03.273075 28431 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.241.126" port: 38479 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:44:03.277102 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 28363
I20260619 02:44:03.277220 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/minicluster-data/master-1/wal/instance
I20260619 02:44:03.277386 6085 external_mini_cluster.cc:1428] Setting key c5b1c6924c79216013ed66b9f7fe1141
Jun 19 02:44:03 dist-test-slave-zpbg krb5kdc[28275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837043, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.125@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:44:03.282392 28433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:03.273724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:38479 (local address 127.0.0.1:34726)
0619 02:44:03.273936 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:03.273980 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:03.274464 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:03.274731 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:03.274734 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:03.275044 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:44:03.275240 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:03.275245 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:03.275924 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:03.275925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:03.276730 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:03.276732 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:03.277125 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:03.277126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:03.277279 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:03.277807 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:44:03.277822 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:44:03.279309 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:44:03.280953 (+ 1644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:03.280961 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:03.280964 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:03.281285 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:03.281501 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:03.281502 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:03.281502 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:03.281597 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:03.281892 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:44:03.281896 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:44:03.282006 (+ 110us) client_negotiation.cc:769] Sending connection context
0619 02:44:03.282087 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 02:44:03.282227 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":87,"threads_started":1}
I20260619 02:44:03.282397 28359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:03.273795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34726 (local address 127.5.241.126:38479)
0619 02:44:03.273869 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:03.273872 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:03.274064 (+ 192us) server_negotiation.cc:407] Connection header received
0619 02:44:03.274570 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:03.274571 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:03.274582 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:03.274625 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:03.275344 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:03.275831 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:03.276820 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:03.277019 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:03.277101 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:03.279401 (+ 2300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:03.279402 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:03.279403 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:03.279411 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:03.280873 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:03.281358 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:03.281359 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:03.281360 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:03.281427 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:03.281655 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:03.281656 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:03.281656 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:03.281825 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:03.281906 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:03.282152 (+ 246us) server_negotiation.cc:299] Negotiation successful
0619 02:44:03.282256 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 02:44:03.283084 28431 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.241.125" port: 35077 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.5.241.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.124@KRBTEST.COM" created.
Jun 19 02:44:03 dist-test-slave-zpbg krb5kdc[28275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837043, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.125@KRBTEST.COM for kudu/127.5.241.125@KRBTEST.COM
I20260619 02:44:03.291165 28433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:03.283367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.125:35077 (local address 127.0.0.1:60442)
0619 02:44:03.283430 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:03.283439 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:03.283470 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:03.283769 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:03.283770 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:03.283901 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:44:03.283942 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:03.283943 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:03.284748 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:03.284749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:03.285341 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:03.285343 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:03.285806 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:03.285807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:03.285874 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:03.286244 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:44:03.286247 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:44:03.288326 (+ 2079us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:44:03.290043 (+ 1717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:03.290044 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:03.290045 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:03.290214 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:03.290409 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:03.290410 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:03.290410 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:03.290469 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:03.290818 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:44:03.290819 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:44:03.290856 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 02:44:03.290933 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 02:44:03.291037 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260619 02:44:03.291181 28435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:03.283432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60442 (local address 127.5.241.125:35077)
0619 02:44:03.283626 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:03.283630 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:03.283659 (+ 29us) server_negotiation.cc:407] Connection header received
0619 02:44:03.283677 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:03.283679 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:03.283690 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:03.283769 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:03.284017 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:03.284668 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:03.285428 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:03.285725 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:03.285807 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:03.288409 (+ 2602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:03.288416 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:03.288416 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:03.288422 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:03.289963 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:03.290282 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:03.290285 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:03.290285 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:03.290336 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:03.290536 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:03.290537 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:03.290538 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:03.290701 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:03.290809 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:03.290977 (+ 168us) server_negotiation.cc:299] Negotiation successful
0619 02:44:03.291088 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":170,"thread_start_us":120,"threads_started":1}
I20260619 02:44:03.291908 28431 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.241.124" port: 41559 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:44:03.292315 28433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:03.292181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.124:41559 (local address 127.0.0.1:38834)
0619 02:44:03.292232 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:03.292285 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.124:41559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260619 02:44:03.292506 28382 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.241.124:41559: connect: Connection refused (error 111)
I20260619 02:44:03.293004 28433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:03.292894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.124:41559 (local address 127.0.0.1:38840)
0619 02:44:03.292940 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:03.292971 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.124:41559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260619 02:44:03.293140 28431 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.241.124:41559: Network error: Client connection negotiation failed: client connection to 127.5.241.124:41559: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.124.keytab.
Entry for principal kudu/127.5.241.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.124.keytab.
I20260619 02:44:03.303678 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.124:41559
--webserver_interface=127.5.241.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.124.keytab
--principal=kudu/127.5.241.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:35911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:38479,127.5.241.125:35077,127.5.241.124:41559
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:44:03.335557 28356 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.241.125" port: 35077 } attempt: 3
Jun 19 02:44:03 dist-test-slave-zpbg krb5kdc[28275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837043, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for kudu/127.5.241.125@KRBTEST.COM
I20260619 02:44:03.342346 28431 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.241.124" port: 41559 } attempt: 1
I20260619 02:44:03.342743 28433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:03.342595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.124:41559 (local address 127.0.0.1:38854)
0619 02:44:03.342671 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:03.342707 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.124:41559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 02:44:03.343129 28358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:03.335930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.125:35077 (local address 127.0.0.1:60454)
0619 02:44:03.335992 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:03.336005 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:03.336076 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:03.336239 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:03.336240 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:03.336401 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:44:03.336466 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:03.336467 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:03.337192 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:03.337193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:03.337723 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:03.337724 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:03.338129 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:03.338130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:03.338212 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:03.338667 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:44:03.338672 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:44:03.340325 (+ 1653us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:44:03.341887 (+ 1562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:03.341888 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:03.341888 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:03.342099 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:03.342287 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:03.342288 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:03.342288 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:03.342370 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:03.342764 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:44:03.342765 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:44:03.342833 (+ 68us) client_negotiation.cc:769] Sending connection context
0619 02:44:03.342937 (+ 104us) client_negotiation.cc:240] Negotiation successful
0619 02:44:03.343032 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260619 02:44:03.343128 28433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:03.343039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.124:41559 (local address 127.0.0.1:38868)
0619 02:44:03.343073 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:03.343106 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.124:41559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260619 02:44:03.343129 28435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:03.336008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60454 (local address 127.5.241.125:35077)
0619 02:44:03.336074 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:03.336076 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:03.336088 (+ 12us) server_negotiation.cc:407] Connection header received
0619 02:44:03.336146 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:03.336147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:03.336159 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:03.336220 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:03.336546 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:03.337107 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:03.337806 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:03.338035 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:03.338128 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:03.340410 (+ 2282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:03.340412 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:03.340412 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:03.340419 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:03.341806 (+ 1387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:03.342167 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:03.342168 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:03.342168 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:03.342226 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:03.342442 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:03.342443 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:03.342444 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:03.342695 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:03.342771 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:03.342948 (+ 177us) server_negotiation.cc:299] Negotiation successful
0619 02:44:03.343029 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260619 02:44:03.343367 28431 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.241.124:41559: Network error: Client connection negotiation failed: client connection to 127.5.241.124:41559: connect: Connection refused (error 111)
I20260619 02:44:03.344095 28356 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.241.124" port: 41559 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:44:03.344481 28358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:03.344386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.124:41559 (local address 127.0.0.1:38872)
0619 02:44:03.344430 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:03.344452 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.124:41559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 02:44:03.344856 28358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:03.344756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.124:41559 (local address 127.0.0.1:38876)
0619 02:44:03.344808 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:03.344836 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.124:41559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260619 02:44:03.344971 28356 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.241.124:41559: Network error: Client connection negotiation failed: client connection to 127.5.241.124:41559: connect: Connection refused (error 111)
I20260619 02:44:03.376120 28356 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.241.124" port: 41559 } attempt: 1
I20260619 02:44:03.376684 28358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:03.376515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.124:41559 (local address 127.0.0.1:38886)
0619 02:44:03.376605 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:03.376651 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.124:41559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260619 02:44:03.377102 28358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:03.377010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.124:41559 (local address 127.0.0.1:38902)
0619 02:44:03.377055 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:03.377084 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.124:41559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260619 02:44:03.377239 28356 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.241.124:41559: Network error: Client connection negotiation failed: client connection to 127.5.241.124:41559: connect: Connection refused (error 111)
W20260619 02:44:03.385648 28438 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 02:44:03.385862 28438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:44:03.385881 28438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:44:03.385923 28438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:44:03.387126 28438 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:44:03.387171 28438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:44:03.387187 28438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:44:03.387199 28438 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:44:03.387240 28438 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:44:03.388688 28438 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:35911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:38479,127.5.241.125:35077,127.5.241.124:41559
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.124.keytab
--principal=kudu/127.5.241.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.124:41559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.241.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28438
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:44:03.388928 28438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:44:03.389149 28438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:44:03.394006 28444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:03.394075 28446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:03.394112 28438 server_base.cc:1034] running on GCE node
W20260619 02:44:03.394091 28443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:03.394562 28438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:44:03.394891 28438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:44:03.396049 28438 hybrid_clock.cc:648] HybridClock initialized: now 1781837043396045 us; error 37 us; skew 500 ppm
Jun 19 02:44:03 dist-test-slave-zpbg krb5kdc[28275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837043, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:44:03.398555 28438 init.cc:377] Logged in from keytab as kudu/127.5.241.124@KRBTEST.COM (short username kudu)
I20260619 02:44:03.399045 28438 webserver.cc:460] Webserver started at http://127.5.241.124:41379/ using document root <none> and password file <none>
I20260619 02:44:03.399271 28438 fs_manager.cc:359] Metadata directory not provided
I20260619 02:44:03.399335 28438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:44:03.399477 28438 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:44:03.400575 28438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/minicluster-data/master-2/data/instance:
uuid: "aa26d398c4e04e208fdd9711b77d2e22"
format_stamp: "Formatted at 2026-06-19 02:44:03 on dist-test-slave-zpbg"
server_key: "71cc787580a474a127efe979785b64c1"
server_key_iv: "e543afbd443421ad9d0a6ca72239fff8"
server_key_version: "encryptionkey@0"
I20260619 02:44:03.400949 28438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/minicluster-data/master-2/wal/instance:
uuid: "aa26d398c4e04e208fdd9711b77d2e22"
format_stamp: "Formatted at 2026-06-19 02:44:03 on dist-test-slave-zpbg"
server_key: "71cc787580a474a127efe979785b64c1"
server_key_iv: "e543afbd443421ad9d0a6ca72239fff8"
server_key_version: "encryptionkey@0"
I20260619 02:44:03.402263 28438 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:44:03.403023 28453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:03.403187 28438 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:44:03.403283 28438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/minicluster-data/master-2/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/minicluster-data/master-2/wal
uuid: "aa26d398c4e04e208fdd9711b77d2e22"
format_stamp: "Formatted at 2026-06-19 02:44:03 on dist-test-slave-zpbg"
server_key: "71cc787580a474a127efe979785b64c1"
server_key_iv: "e543afbd443421ad9d0a6ca72239fff8"
server_key_version: "encryptionkey@0"
I20260619 02:44:03.403352 28438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:44:03.411507 28431 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.241.124" port: 41559 } attempt: 2
I20260619 02:44:03.411948 28433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:03.411821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.124:41559 (local address 127.0.0.1:38916)
0619 02:44:03.411880 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:03.411919 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.124:41559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 02:44:03.412276 28433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:03.412202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.124:41559 (local address 127.0.0.1:38926)
0619 02:44:03.412237 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:03.412261 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.124:41559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260619 02:44:03.412384 28431 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.241.124:41559: Network error: Client connection negotiation failed: client connection to 127.5.241.124:41559: connect: Connection refused (error 111)
I20260619 02:44:03.422506 28438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:44:03.424634 28438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:44:03.424790 28438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:44:03.428785 28438 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.124:41559
I20260619 02:44:03.428856 28505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.124:41559 every 8 connection(s)
I20260619 02:44:03.429190 28438 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/minicluster-data/master-2/data/info.pb
I20260619 02:44:03.430172 28506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:44:03.431785 28506 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.241.126" port: 38479 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:44:03 dist-test-slave-zpbg krb5kdc[28275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837043, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.124@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:44:03.438295 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 28438
I20260619 02:44:03.438401 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836772852275-6085-0/minicluster-data/master-2/wal/instance
I20260619 02:44:03.438611 6085 external_mini_cluster.cc:1428] Setting key 5be6525faa8e5e8b0dc5c35352714eeb
I20260619 02:44:03.440838 28359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:03.432438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34740 (local address 127.5.241.126:38479)
0619 02:44:03.432486 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:03.432488 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:03.432685 (+ 197us) server_negotiation.cc:407] Connection header received
0619 02:44:03.433136 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:03.433137 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:03.433146 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:03.433192 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:03.433799 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:03.434258 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:03.435269 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:03.435445 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:03.435498 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:03.437660 (+ 2162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:03.437661 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:03.437662 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:03.437668 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:03.439286 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:03.439748 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:03.439750 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:03.439750 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:03.439834 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:03.440158 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:03.440158 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:03.440159 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:03.440320 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:03.440398 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:03.440666 (+ 268us) server_negotiation.cc:299] Negotiation successful
0619 02:44:03.440746 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 02:44:03.440884 28508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:03.432404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:38479 (local address 127.0.0.1:34740)
0619 02:44:03.432569 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:03.432609 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:03.433036 (+ 427us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:03.433263 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:03.433266 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:03.433545 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:44:03.433730 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:03.433734 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:03.434331 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:03.434332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:03.435195 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:03.435197 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:03.435546 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:03.435546 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:03.435647 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:03.436152 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:44:03.436161 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:44:03.437575 (+ 1414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:44:03.439378 (+ 1803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:03.439383 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:03.439385 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:03.439686 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:03.439929 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:03.439929 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:03.439930 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:03.440074 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:03.440407 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:44:03.440413 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:44:03.440530 (+ 117us) client_negotiation.cc:769] Sending connection context
0619 02:44:03.440616 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 02:44:03.440730 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":89,"threads_started":1}
I20260619 02:44:03.441444 28506 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.241.125" port: 35077 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:44:03 dist-test-slave-zpbg krb5kdc[28275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837042, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.241.124@KRBTEST.COM
Jun 19 02:44:03 dist-test-slave-zpbg krb5kdc[28275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837043, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.124@KRBTEST.COM for kudu/127.5.241.125@KRBTEST.COM
I20260619 02:44:03.448041 28509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:03.439031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38936 (local address 127.5.241.124:41559)
0619 02:44:03.439176 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:03.439182 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:03.439324 (+ 142us) server_negotiation.cc:407] Connection header received
0619 02:44:03.439380 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:03.439382 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:03.439397 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:03.439494 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:03.440297 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:03.441337 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:03.442245 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:03.442567 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:03.442655 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:03.445024 (+ 2369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:03.445032 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:03.445032 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:03.445039 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:03.446814 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:03.447208 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:03.447210 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:03.447211 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:03.447263 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:03.447455 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:03.447456 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:03.447456 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:03.447581 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:03.447679 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:03.447817 (+ 138us) server_negotiation.cc:299] Negotiation successful
0619 02:44:03.447899 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":90,"threads_started":1}
I20260619 02:44:03.448804 28508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:03.441717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.125:35077 (local address 127.0.0.1:60458)
0619 02:44:03.441763 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:03.441773 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:03.441809 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:03.441928 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:03.441929 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:03.442059 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:44:03.442102 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:03.442103 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:03.442714 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:03.442715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:03.443312 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:03.443314 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:03.443716 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:03.443717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:03.443785 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:03.444782 (+ 997us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:44:03.444786 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:44:03.446100 (+ 1314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:44:03.447764 (+ 1664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:03.447765 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:03.447765 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:03.447987 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:03.448197 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:03.448197 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:03.448197 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:03.448249 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:03.448520 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:44:03.448521 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:44:03.448562 (+ 41us) client_negotiation.cc:769] Sending connection context
0619 02:44:03.448629 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 02:44:03.448714 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260619 02:44:03.448808 28435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:03.441789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60458 (local address 127.5.241.125:35077)
0619 02:44:03.441835 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:03.441837 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:03.441845 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:44:03.441870 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:03.441871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:03.441880 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:03.441928 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:03.442195 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:03.442638 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:03.443396 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:03.443635 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:03.443715 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:03.446181 (+ 2466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:03.446182 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:03.446182 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:03.446188 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:03.447688 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:03.448073 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:03.448074 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:03.448074 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:03.448130 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:03.448307 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:03.448308 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:03.448308 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:03.448430 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:03.448482 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:03.448655 (+ 173us) server_negotiation.cc:299] Negotiation successful
0619 02:44:03.448720 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 02:44:03.449352 28506 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.241.124" port: 41559 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:44:03.452396 28356 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.241.124" port: 41559 } attempt: 2
Jun 19 02:44:03 dist-test-slave-zpbg krb5kdc[28275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837042, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
Jun 19 02:44:03 dist-test-slave-zpbg krb5kdc[28275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837043, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.124@KRBTEST.COM for kudu/127.5.241.124@KRBTEST.COM
Jun 19 02:44:03 dist-test-slave-zpbg krb5kdc[28275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837043, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for kudu/127.5.241.124@KRBTEST.COM
I20260619 02:44:03.457792 28359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:03.448831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34742 (local address 127.5.241.126:38479)
0619 02:44:03.449064 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:03.449065 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:03.449071 (+ 6us) server_negotiation.cc:407] Connection header received
0619 02:44:03.449089 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:03.449090 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:03.449097 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:03.449162 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:03.449700 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:03.450163 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:03.450866 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:03.451146 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:03.451243 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:03.454192 (+ 2949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:03.454193 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:03.454193 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:03.454200 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:03.456527 (+ 2327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:03.456950 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:03.456951 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:03.456951 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:03.457007 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:03.457233 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:03.457235 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:03.457235 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:03.457369 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:03.457558 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:03.457600 (+ 42us) server_negotiation.cc:299] Negotiation successful
0619 02:44:03.457673 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216}
I20260619 02:44:03.458374 28508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:03.449621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.124:41559 (local address 127.0.0.1:38944)
0619 02:44:03.449664 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:03.449675 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:03.449712 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:03.449834 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:03.449834 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:03.449956 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:44:03.449978 (+ 22us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:03.449979 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:03.450701 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:03.450702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:03.451403 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:03.451406 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:03.453699 (+ 2293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:03.453700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:03.453753 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:03.454142 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:44:03.454146 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:44:03.455465 (+ 1319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:44:03.457407 (+ 1942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:03.457408 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:03.457408 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:03.457585 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:03.457763 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:03.457763 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:03.457764 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:03.457807 (+ 43us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:03.458064 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:44:03.458065 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:44:03.458098 (+ 33us) client_negotiation.cc:769] Sending connection context
0619 02:44:03.458163 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 02:44:03.458267 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260619 02:44:03.458665 28509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:03.449678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38944 (local address 127.5.241.124:41559)
0619 02:44:03.449706 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:03.449708 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:03.449715 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:44:03.449768 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:03.449769 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:03.449778 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:03.449829 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:03.450056 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:03.450620 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:03.451483 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:03.453598 (+ 2115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:03.453686 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:03.455553 (+ 1867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:03.455554 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:03.455554 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:03.455561 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:03.457333 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:03.457659 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:03.457660 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:03.457660 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:03.457706 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:03.457867 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:03.457867 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:03.457868 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:03.457979 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:03.458043 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:03.458196 (+ 153us) server_negotiation.cc:299] Negotiation successful
0619 02:44:03.458336 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":14}
I20260619 02:44:03.460316 28506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa26d398c4e04e208fdd9711b77d2e22: Bootstrap starting.
I20260619 02:44:03.460594 28358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:03.452700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.124:41559 (local address 127.0.0.1:38946)
0619 02:44:03.452746 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:03.452757 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:03.452792 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:03.453104 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:03.453105 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:03.453311 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:44:03.453349 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:03.453351 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:03.454259 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:03.454260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:03.455105 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:03.455108 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:03.455527 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:03.455528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:03.455591 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:03.455963 (+ 372us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:44:03.455968 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:44:03.457384 (+ 1416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:44:03.459600 (+ 2216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:03.459601 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:03.459601 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:03.459769 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:03.459962 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:03.459963 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:03.459963 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:03.460028 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:03.460284 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:44:03.460285 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:44:03.460328 (+ 43us) client_negotiation.cc:769] Sending connection context
0619 02:44:03.460409 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 02:44:03.460497 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260619 02:44:03.460654 28511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:03.452765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38946 (local address 127.5.241.124:41559)
0619 02:44:03.452985 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:03.452987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:03.452994 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:44:03.453012 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:03.453013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:03.453021 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:03.453083 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:03.453435 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:03.454153 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:03.455199 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:03.455441 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:03.455514 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:03.457855 (+ 2341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:03.457856 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:03.457857 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:03.457863 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:03.459521 (+ 1658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:03.459838 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:03.459839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:03.459839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:03.459891 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:03.460093 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:03.460094 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:03.460094 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:03.460217 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:03.460293 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:03.460458 (+ 165us) server_negotiation.cc:299] Negotiation successful
0619 02:44:03.460540 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":160,"threads_started":1}
I20260619 02:44:03.461041 28506 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa26d398c4e04e208fdd9711b77d2e22: Neither blocks nor log segments found. Creating new log.
I20260619 02:44:03.461344 28506 log.cc:824] T 00000000000000000000000000000000 P aa26d398c4e04e208fdd9711b77d2e22: Log is configured to *not* fsync() on all Append() calls
I20260619 02:44:03.462314 28506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa26d398c4e04e208fdd9711b77d2e22: No bootstrap required, opened a new log
I20260619 02:44:03.462342 28356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 910d082c01904599ad1bacbe07a76dce: Bootstrap starting.
I20260619 02:44:03.463074 28356 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 910d082c01904599ad1bacbe07a76dce: Neither blocks nor log segments found. Creating new log.
I20260619 02:44:03.463387 28356 log.cc:824] T 00000000000000000000000000000000 P 910d082c01904599ad1bacbe07a76dce: Log is configured to *not* fsync() on all Append() calls
Jun 19 02:44:03 dist-test-slave-zpbg krb5kdc[28275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837042, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.241.125@KRBTEST.COM
I20260619 02:44:03.464174 28506 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa26d398c4e04e208fdd9711b77d2e22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "910d082c01904599ad1bacbe07a76dce" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 38479 } } peers { permanent_uuid: "52f896763048436482043ee9a516226d" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 35077 } } peers { permanent_uuid: "aa26d398c4e04e208fdd9711b77d2e22" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 41559 } }
I20260619 02:44:03.464259 28356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 910d082c01904599ad1bacbe07a76dce: No bootstrap required, opened a new log
I20260619 02:44:03.464313 28506 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa26d398c4e04e208fdd9711b77d2e22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa26d398c4e04e208fdd9711b77d2e22, State: Initialized, Role: FOLLOWER
I20260619 02:44:03.464416 28506 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa26d398c4e04e208fdd9711b77d2e22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "910d082c01904599ad1bacbe07a76dce" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 38479 } } peers { permanent_uuid: "52f896763048436482043ee9a516226d" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 35077 } } peers { permanent_uuid: "aa26d398c4e04e208fdd9711b77d2e22" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 41559 } }
I20260619 02:44:03.464669 28514 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa26d398c4e04e208fdd9711b77d2e22 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "910d082c01904599ad1bacbe07a76dce" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 38479 } } peers { permanent_uuid: "52f896763048436482043ee9a516226d" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 35077 } } peers { permanent_uuid: "aa26d398c4e04e208fdd9711b77d2e22" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 41559 } } }
I20260619 02:44:03.464718 28506 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa26d398c4e04e208fdd9711b77d2e22 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:44:03.464800 28514 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa26d398c4e04e208fdd9711b77d2e22 [sys.catalog]: This master's current role is: FOLLOWER
W20260619 02:44:03.466693 28525 catalog_manager.cc:1558] T 00000000000000000000000000000000 P aa26d398c4e04e208fdd9711b77d2e22: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:44:03.466810 28525 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:44:03.466952 28356 raft_consensus.cc:348] T 00000000000000000000000000000000 P 910d082c01904599ad1bacbe07a76dce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "910d082c01904599ad1bacbe07a76dce" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 38479 } } peers { permanent_uuid: "52f896763048436482043ee9a516226d" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 35077 } } peers { permanent_uuid: "aa26d398c4e04e208fdd9711b77d2e22" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 41559 } }
I20260619 02:44:03.467176 28356 raft_consensus.cc:729] T 00000000000000000000000000000000 P 910d082c01904599ad1bacbe07a76dce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 910d082c01904599ad1bacbe07a76dce, State: Initialized, Role: FOLLOWER
I20260619 02:44:03.467368 28435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:03.459053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60472 (local address 127.5.241.125:35077)
0619 02:44:03.459094 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:03.459095 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:03.459106 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:44:03.459131 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:03.459132 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:03.459141 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:03.459207 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:03.459789 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:03.460443 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:03.461365 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:03.461657 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:03.461733 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:03.464333 (+ 2600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:03.464335 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:03.464335 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:03.464342 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:03.466097 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:03.466484 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:03.466486 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:03.466486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:03.466548 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:03.466815 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:03.466816 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:03.466816 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:03.466953 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:03.467144 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:03.467180 (+ 36us) server_negotiation.cc:299] Negotiation successful
0619 02:44:03.467253 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:44:03.467315 28356 consensus_queue.cc:260] T 00000000000000000000000000000000 P 910d082c01904599ad1bacbe07a76dce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "910d082c01904599ad1bacbe07a76dce" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 38479 } } peers { permanent_uuid: "52f896763048436482043ee9a516226d" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 35077 } } peers { permanent_uuid: "aa26d398c4e04e208fdd9711b77d2e22" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 41559 } }
I20260619 02:44:03.467764 28356 sys_catalog.cc:564] T 00000000000000000000000000000000 P 910d082c01904599ad1bacbe07a76dce [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:44:03.468606 6085 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:44:03.469585 28330 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "910d082c01904599ad1bacbe07a76dce"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34742
W20260619 02:44:03.469641 28538 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 910d082c01904599ad1bacbe07a76dce: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:44:03.469681 28538 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:44:03.469699 28330 raft_consensus.cc:482] T 00000000000000000000000000000000 P 910d082c01904599ad1bacbe07a76dce [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 02:44:03.469746 28330 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 910d082c01904599ad1bacbe07a76dce [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:44:03.466864 28438 master_runner.cc:186] Error getting master registration for 127.5.241.126:38479: OK, instance_id { permanent_uuid: "910d082c01904599ad1bacbe07a76dce" instance_seqno: 1781837043108994 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 02:44:03.469825 28438 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 02:44:03.470594 28330 raft_consensus.cc:504] T 00000000000000000000000000000000 P 910d082c01904599ad1bacbe07a76dce [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "910d082c01904599ad1bacbe07a76dce" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 38479 } } peers { permanent_uuid: "52f896763048436482043ee9a516226d" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 35077 } } peers { permanent_uuid: "aa26d398c4e04e208fdd9711b77d2e22" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 41559 } }
I20260619 02:44:03.471033 28330 leader_election.cc:288] T 00000000000000000000000000000000 P 910d082c01904599ad1bacbe07a76dce [CANDIDATE]: Term 1 election: Requested vote from peers 52f896763048436482043ee9a516226d (127.5.241.125:35077), aa26d398c4e04e208fdd9711b77d2e22 (127.5.241.124:41559)
I20260619 02:44:03.470189 28288 master_runner.cc:186] Error getting master registration for 127.5.241.125:35077: OK, instance_id { permanent_uuid: "52f896763048436482043ee9a516226d" instance_seqno: 1781837043269037 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 02:44:03.472697 28288 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 02:44:03.472831 28480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "910d082c01904599ad1bacbe07a76dce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "aa26d398c4e04e208fdd9711b77d2e22"
I20260619 02:44:03.472922 28480 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa26d398c4e04e208fdd9711b77d2e22 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:44:03.473770 28480 raft_consensus.cc:2454] T 00000000000000000000000000000000 P aa26d398c4e04e208fdd9711b77d2e22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 910d082c01904599ad1bacbe07a76dce in term 1.
I20260619 02:44:03.474182 28307 leader_election.cc:302] T 00000000000000000000000000000000 P 910d082c01904599ad1bacbe07a76dce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 910d082c01904599ad1bacbe07a76dce, aa26d398c4e04e208fdd9711b77d2e22; no voters:
I20260619 02:44:03.474571 28527 sys_catalog.cc:455] T 00000000000000000000000000000000 P 910d082c01904599ad1bacbe07a76dce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "910d082c01904599ad1bacbe07a76dce" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 38479 } } peers { permanent_uuid: "52f896763048436482043ee9a516226d" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 35077 } } peers { permanent_uuid: "aa26d398c4e04e208fdd9711b77d2e22" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 41559 } } }
I20260619 02:44:03.474720 28527 sys_catalog.cc:458] T 00000000000000000000000000000000 P 910d082c01904599ad1bacbe07a76dce [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:44:03.474834 28527 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 910d082c01904599ad1bacbe07a76dce [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:44:03.474937 28527 raft_consensus.cc:686] T 00000000000000000000000000000000 P 910d082c01904599ad1bacbe07a76dce [term 1 LEADER]: Becoming Leader. State: Replica: 910d082c01904599ad1bacbe07a76dce, State: Running, Role: LEADER
I20260619 02:44:03.475054 28527 consensus_queue.cc:237] T 00000000000000000000000000000000 P 910d082c01904599ad1bacbe07a76dce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "910d082c01904599ad1bacbe07a76dce" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 38479 } } peers { permanent_uuid: "52f896763048436482043ee9a516226d" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 35077 } } peers { permanent_uuid: "aa26d398c4e04e208fdd9711b77d2e22" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 41559 } }
I20260619 02:44:03.475581 28527 sys_catalog.cc:455] T 00000000000000000000000000000000 P 910d082c01904599ad1bacbe07a76dce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 910d082c01904599ad1bacbe07a76dce. Latest consensus state: current_term: 1 leader_uuid: "910d082c01904599ad1bacbe07a76dce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "910d082c01904599ad1bacbe07a76dce" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 38479 } } peers { permanent_uuid: "52f896763048436482043ee9a516226d" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 35077 } } peers { permanent_uuid: "aa26d398c4e04e208fdd9711b77d2e22" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 41559 } } }
I20260619 02:44:03.475728 28527 sys_catalog.cc:458] T 00000000000000000000000000000000 P 910d082c01904599ad1bacbe07a76dce [sys.catalog]: This master's current role is: LEADER
I20260619 02:44:03.475967 28548 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:44:03.476199 28548 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:44:03.478190 28480 raft_consensus.cc:1261] T 00000000000000000000000000000000 P aa26d398c4e04e208fdd9711b77d2e22 [term 1 FOLLOWER]: Refusing update from remote peer 910d082c01904599ad1bacbe07a76dce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:44:03.478556 28546 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 910d082c01904599ad1bacbe07a76dce [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa26d398c4e04e208fdd9711b77d2e22" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 41559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:44:03.479924 28514 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa26d398c4e04e208fdd9711b77d2e22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 910d082c01904599ad1bacbe07a76dce. Latest consensus state: current_term: 1 leader_uuid: "910d082c01904599ad1bacbe07a76dce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "910d082c01904599ad1bacbe07a76dce" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 38479 } } peers { permanent_uuid: "52f896763048436482043ee9a516226d" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 35077 } } peers { permanent_uuid: "aa26d398c4e04e208fdd9711b77d2e22" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 41559 } } }
I20260619 02:44:03.480008 28514 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa26d398c4e04e208fdd9711b77d2e22 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:44:03.481599 28546 sys_catalog.cc:455] T 00000000000000000000000000000000 P 910d082c01904599ad1bacbe07a76dce [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "910d082c01904599ad1bacbe07a76dce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "910d082c01904599ad1bacbe07a76dce" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 38479 } } peers { permanent_uuid: "52f896763048436482043ee9a516226d" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 35077 } } peers { permanent_uuid: "aa26d398c4e04e208fdd9711b77d2e22" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 41559 } } }
I20260619 02:44:03.481673 28546 sys_catalog.cc:458] T 00000000000000000000000000000000 P 910d082c01904599ad1bacbe07a76dce [sys.catalog]: This master's current role is: LEADER
I20260619 02:44:03.482128 28514 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa26d398c4e04e208fdd9711b77d2e22 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "910d082c01904599ad1bacbe07a76dce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "910d082c01904599ad1bacbe07a76dce" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 38479 } } peers { permanent_uuid: "52f896763048436482043ee9a516226d" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 35077 } } peers { permanent_uuid: "aa26d398c4e04e208fdd9711b77d2e22" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 41559 } } }
I20260619 02:44:03.482218 28514 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa26d398c4e04e208fdd9711b77d2e22 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:44:03.482542 28549 mvcc.cc:205] Tried to move back new op lower bound from 7298404530084306944 to 7298404530074857472. Current Snapshot: MvccSnapshot[applied={T|T < 7298404530084306944}]
I20260619 02:44:03.483500 28551 mvcc.cc:205] Tried to move back new op lower bound from 7298404530084306944 to 7298404530074857472. Current Snapshot: MvccSnapshot[applied={T|T < 7298404530084306944}]
I20260619 02:44:03.484294 28548 catalog_manager.cc:1347] Generated new cluster ID: 484adabd9fab4c11b12ac2ced4256c19
I20260619 02:44:03.484339 28548 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:44:03.491192 28435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:03.472417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60486 (local address 127.5.241.125:35077)
0619 02:44:03.472452 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:03.472454 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:03.472681 (+ 227us) server_negotiation.cc:407] Connection header received
0619 02:44:03.472809 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:03.472810 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:03.472820 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:03.472849 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:03.474008 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:03.474726 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:03.476273 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:03.477000 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:03.477046 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:03.485901 (+ 8855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:03.485903 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:03.485904 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:03.485913 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:03.487695 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:03.489196 (+ 1501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:03.489197 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:03.489198 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:03.489269 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:03.490055 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:03.490056 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:03.490057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:03.490218 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:03.490292 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:03.490994 (+ 702us) server_negotiation.cc:299] Negotiation successful
0619 02:44:03.491077 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 02:44:03.498498 28511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:03.478966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38952 (local address 127.5.241.124:41559)
0619 02:44:03.479008 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:03.479010 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:03.479404 (+ 394us) server_negotiation.cc:407] Connection header received
0619 02:44:03.479498 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:03.479499 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:03.479511 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:03.479543 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:03.480532 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:03.481267 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:03.482907 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:03.483265 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:03.483311 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:03.493526 (+ 10215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:03.493528 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:03.493528 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:03.493538 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:03.495399 (+ 1861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:03.496795 (+ 1396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:03.496797 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:03.496797 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:03.496867 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:03.497404 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:03.497405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:03.497406 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:03.497597 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:03.497692 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:03.498312 (+ 620us) server_negotiation.cc:299] Negotiation successful
0619 02:44:03.498398 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:44:03.503530 28431 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.241.124" port: 41559 } attempt: 3
I20260619 02:44:03.503769 28359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:03.472330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34746 (local address 127.5.241.126:38479)
0619 02:44:03.472366 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:03.472368 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:03.479260 (+ 6892us) server_negotiation.cc:407] Connection header received
0619 02:44:03.479332 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:03.479332 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:03.479353 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:03.479402 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:03.484731 (+ 5329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:03.485451 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:03.493035 (+ 7584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:03.493362 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:03.493405 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:03.500291 (+ 6886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:03.500293 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:03.500294 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:03.500302 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:03.502065 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:03.502674 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:03.502676 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:03.502676 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:03.502734 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:03.503033 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:03.503034 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:03.503034 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:03.503185 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:03.503280 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:03.503589 (+ 309us) server_negotiation.cc:299] Negotiation successful
0619 02:44:03.503672 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
Jun 19 02:44:03 dist-test-slave-zpbg krb5kdc[28275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837043, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.125@KRBTEST.COM for kudu/127.5.241.124@KRBTEST.COM
I20260619 02:44:03.512349 28433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:03.503840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.124:41559 (local address 127.0.0.1:38960)
0619 02:44:03.503969 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:03.503981 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:03.504021 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:03.504534 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:03.504535 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:03.504713 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:44:03.504760 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:03.504762 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:03.505733 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:03.505734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:03.506612 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:03.506615 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:03.507181 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:03.507182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:03.507257 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:03.507652 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:44:03.507657 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:44:03.509147 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:44:03.511171 (+ 2024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:03.511172 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:03.511173 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:03.511388 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:03.511624 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:03.511624 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:03.511625 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:03.511693 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:03.512023 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:44:03.512024 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:44:03.512076 (+ 52us) client_negotiation.cc:769] Sending connection context
0619 02:44:03.512149 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 02:44:03.512245 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108}
I20260619 02:44:03.512737 28511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:03.504096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38960 (local address 127.5.241.124:41559)
0619 02:44:03.504134 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:03.504136 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:03.504142 (+ 6us) server_negotiation.cc:407] Connection header received
0619 02:44:03.504162 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:03.504163 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:03.504172 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:03.504216 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:03.504852 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:03.505614 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:03.506718 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:03.507044 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:03.507103 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:03.509248 (+ 2145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:03.509250 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:03.509250 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:03.509256 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:03.511063 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:03.511471 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:03.511472 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:03.511472 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:03.511546 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:03.511765 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:03.511766 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:03.511766 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:03.511913 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:03.511970 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:03.512566 (+ 596us) server_negotiation.cc:299] Negotiation successful
0619 02:44:03.512647 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 02:44:03.514236 28431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52f896763048436482043ee9a516226d: Bootstrap starting.
I20260619 02:44:03.514961 28431 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 52f896763048436482043ee9a516226d: Neither blocks nor log segments found. Creating new log.
I20260619 02:44:03.515277 28431 log.cc:824] T 00000000000000000000000000000000 P 52f896763048436482043ee9a516226d: Log is configured to *not* fsync() on all Append() calls
I20260619 02:44:03.522630 28548 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:44:03.524142 28548 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:44:03.527524 28431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52f896763048436482043ee9a516226d: No bootstrap required, opened a new log
I20260619 02:44:03.529624 28431 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52f896763048436482043ee9a516226d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "910d082c01904599ad1bacbe07a76dce" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 38479 } } peers { permanent_uuid: "52f896763048436482043ee9a516226d" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 35077 } } peers { permanent_uuid: "aa26d398c4e04e208fdd9711b77d2e22" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 41559 } }
I20260619 02:44:03.529783 28431 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52f896763048436482043ee9a516226d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52f896763048436482043ee9a516226d, State: Initialized, Role: FOLLOWER
I20260619 02:44:03.529872 28431 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52f896763048436482043ee9a516226d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "910d082c01904599ad1bacbe07a76dce" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 38479 } } peers { permanent_uuid: "52f896763048436482043ee9a516226d" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 35077 } } peers { permanent_uuid: "aa26d398c4e04e208fdd9711b77d2e22" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 41559 } }
I20260619 02:44:03.530225 28431 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52f896763048436482043ee9a516226d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:44:03.530340 28404 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52f896763048436482043ee9a516226d [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:44:03.531311 28404 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 52f896763048436482043ee9a516226d [term 1 FOLLOWER]: Refusing update from remote peer 910d082c01904599ad1bacbe07a76dce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:44:03.531690 28405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "910d082c01904599ad1bacbe07a76dce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "52f896763048436482043ee9a516226d"
I20260619 02:44:03.531715 28546 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 910d082c01904599ad1bacbe07a76dce [LEADER]: Connected to new peer: Peer: permanent_uuid: "52f896763048436482043ee9a516226d" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 35077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:44:03.532595 28405 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 52f896763048436482043ee9a516226d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 910d082c01904599ad1bacbe07a76dce in term 1.
W20260619 02:44:03.534873 28567 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 52f896763048436482043ee9a516226d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:44:03.534929 28567 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:44:03.537644 28568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52f896763048436482043ee9a516226d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "910d082c01904599ad1bacbe07a76dce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "910d082c01904599ad1bacbe07a76dce" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 38479 } } peers { permanent_uuid: "52f896763048436482043ee9a516226d" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 35077 } } peers { permanent_uuid: "aa26d398c4e04e208fdd9711b77d2e22" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 41559 } } }
I20260619 02:44:03.537784 28568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52f896763048436482043ee9a516226d [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:44:03.537875 28568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52f896763048436482043ee9a516226d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 910d082c01904599ad1bacbe07a76dce. Latest consensus state: current_term: 1 leader_uuid: "910d082c01904599ad1bacbe07a76dce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "910d082c01904599ad1bacbe07a76dce" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 38479 } } peers { permanent_uuid: "52f896763048436482043ee9a516226d" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 35077 } } peers { permanent_uuid: "aa26d398c4e04e208fdd9711b77d2e22" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 41559 } } }
I20260619 02:44:03.537968 28568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52f896763048436482043ee9a516226d [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:44:03.538034 28568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52f896763048436482043ee9a516226d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "910d082c01904599ad1bacbe07a76dce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "910d082c01904599ad1bacbe07a76dce" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 38479 } } peers { permanent_uuid: "52f896763048436482043ee9a516226d" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 35077 } } peers { permanent_uuid: "aa26d398c4e04e208fdd9711b77d2e22" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 41559 } } }
I20260619 02:44:03.538122 28568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52f896763048436482043ee9a516226d [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:44:03.539217 28569 mvcc.cc:205] Tried to move back new op lower bound from 7298404530263191552 to 7298404530074857472. Current Snapshot: MvccSnapshot[applied={T|T < 7298404530084306944}]
I20260619 02:44:03.544179 28546 sys_catalog.cc:455] T 00000000000000000000000000000000 P 910d082c01904599ad1bacbe07a76dce [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "910d082c01904599ad1bacbe07a76dce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "910d082c01904599ad1bacbe07a76dce" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 38479 } } peers { permanent_uuid: "52f896763048436482043ee9a516226d" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 35077 } } peers { permanent_uuid: "aa26d398c4e04e208fdd9711b77d2e22" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 41559 } } }
I20260619 02:44:03.544269 28546 sys_catalog.cc:458] T 00000000000000000000000000000000 P 910d082c01904599ad1bacbe07a76dce [sys.catalog]: This master's current role is: LEADER
I20260619 02:44:03.545845 28548 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 910d082c01904599ad1bacbe07a76dce: Generated new TSK 0
I20260619 02:44:03.546058 28548 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 02:44:03.574190 28511 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:44:03.572800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38966 (local address 127.5.241.124:41559)
0619 02:44:03.572846 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:03.572849 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:03.573494 (+ 645us) server_negotiation.cc:407] Connection header received
0619 02:44:03.573617 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:03.573618 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:03.573630 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:03.573662 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:03.574125 (+ 463us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38966: BlockingRecv error: recv got EOF from 127.0.0.1:38966 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260619 02:44:03.574952 28359 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:44:03.573190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34750 (local address 127.5.241.126:38479)
0619 02:44:03.573230 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:03.573232 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:03.574422 (+ 1190us) server_negotiation.cc:407] Connection header received
0619 02:44:03.574521 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:03.574522 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:03.574533 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:03.574562 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:03.574892 (+ 330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34750: BlockingRecv error: recv got EOF from 127.0.0.1:34750 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260619 02:44:03.575995 28435 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:44:03.575327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60500 (local address 127.5.241.125:35077)
0619 02:44:03.575367 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:03.575369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:03.575485 (+ 116us) server_negotiation.cc:407] Connection header received
0619 02:44:03.575581 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:03.575582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:03.575593 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:03.575624 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:03.575933 (+ 309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60500: BlockingRecv error: recv got EOF from 127.0.0.1:60500 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260619 02:44:03.578271 28435 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:44:03.577564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60514 (local address 127.5.241.125:35077)
0619 02:44:03.577597 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:03.577598 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:03.577768 (+ 170us) server_negotiation.cc:407] Connection header received
0619 02:44:03.577868 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:03.577868 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:03.577885 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:03.577914 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:03.578226 (+ 312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60514: BlockingRecv error: recv got EOF from 127.0.0.1:60514 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260619 02:44:03.581212 28435 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:44:03.580460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60526 (local address 127.5.241.125:35077)
0619 02:44:03.580492 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:03.580494 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:03.580743 (+ 249us) server_negotiation.cc:407] Connection header received
0619 02:44:03.580835 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:03.580836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:03.580846 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:03.580873 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:03.581171 (+ 298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60526: BlockingRecv error: recv got EOF from 127.0.0.1:60526 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260619 02:44:03.585091 28435 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:44:03.584363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60542 (local address 127.5.241.125:35077)
0619 02:44:03.584395 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:03.584397 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:03.584601 (+ 204us) server_negotiation.cc:407] Connection header received
0619 02:44:03.584696 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:03.584697 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:03.584706 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:03.584734 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:03.585050 (+ 316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60542: BlockingRecv error: recv got EOF from 127.0.0.1:60542 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260619 02:44:03.591260 28435 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:44:03.590246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60554 (local address 127.5.241.125:35077)
0619 02:44:03.590481 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:03.590482 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:03.590707 (+ 225us) server_negotiation.cc:407] Connection header received
0619 02:44:03.590840 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:03.590841 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:03.590851 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:03.590879 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:03.591213 (+ 334us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60554: BlockingRecv error: recv got EOF from 127.0.0.1:60554 (error 108)
Metrics: {"server-negotiator.queue_time_us":215}
W20260619 02:44:03.601395 28435 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:44:03.600518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60568 (local address 127.5.241.125:35077)
0619 02:44:03.600557 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:03.600559 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:03.600814 (+ 255us) server_negotiation.cc:407] Connection header received
0619 02:44:03.600918 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:03.600919 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:03.600930 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:03.600961 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:03.601337 (+ 376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60568: BlockingRecv error: recv got EOF from 127.0.0.1:60568 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260619 02:44:03.619726 28435 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:44:03.618753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60578 (local address 127.5.241.125:35077)
0619 02:44:03.618827 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:03.618830 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:03.619095 (+ 265us) server_negotiation.cc:407] Connection header received
0619 02:44:03.619221 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:03.619222 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:03.619234 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:03.619264 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:03.619665 (+ 401us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60578: BlockingRecv error: recv got EOF from 127.0.0.1:60578 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260619 02:44:03.653738 28435 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:44:03.653035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60584 (local address 127.5.241.125:35077)
0619 02:44:03.653084 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:03.653086 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:03.653199 (+ 113us) server_negotiation.cc:407] Connection header received
0619 02:44:03.653286 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:03.653287 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:03.653299 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:03.653345 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:03.653673 (+ 328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60584: BlockingRecv error: recv got EOF from 127.0.0.1:60584 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260619 02:44:03.719924 28435 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:44:03.719073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60586 (local address 127.5.241.125:35077)
0619 02:44:03.719177 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:03.719180 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:03.719290 (+ 110us) server_negotiation.cc:407] Connection header received
0619 02:44:03.719374 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:03.719375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:03.719388 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:03.719462 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:03.719847 (+ 385us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60586: BlockingRecv error: recv got EOF from 127.0.0.1:60586 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260619 02:44:03.849856 28435 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:44:03.849047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60596 (local address 127.5.241.125:35077)
0619 02:44:03.849144 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:03.849147 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:03.849243 (+ 96us) server_negotiation.cc:407] Connection header received
0619 02:44:03.849329 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:03.849330 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:03.849345 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:03.849408 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:03.849784 (+ 376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60596: BlockingRecv error: recv got EOF from 127.0.0.1:60596 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260619 02:44:04.107672 28435 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:44:04.106960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60600 (local address 127.5.241.125:35077)
0619 02:44:04.107015 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:04.107017 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:04.107129 (+ 112us) server_negotiation.cc:407] Connection header received
0619 02:44:04.107206 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:04.107207 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:04.107219 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:04.107292 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:04.107609 (+ 317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60600: BlockingRecv error: recv got EOF from 127.0.0.1:60600 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 02:44:04.467296 28525 catalog_manager.cc:1259] Loaded cluster ID: 484adabd9fab4c11b12ac2ced4256c19
I20260619 02:44:04.467368 28525 catalog_manager.cc:1552] T 00000000000000000000000000000000 P aa26d398c4e04e208fdd9711b77d2e22: loading cluster ID for follower catalog manager: success
I20260619 02:44:04.468258 28525 catalog_manager.cc:1574] T 00000000000000000000000000000000 P aa26d398c4e04e208fdd9711b77d2e22: acquiring CA information for follower catalog manager: success
I20260619 02:44:04.468511 28525 catalog_manager.cc:1602] T 00000000000000000000000000000000 P aa26d398c4e04e208fdd9711b77d2e22: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 02:44:04.535365 28567 catalog_manager.cc:1259] Loaded cluster ID: 484adabd9fab4c11b12ac2ced4256c19
I20260619 02:44:04.535440 28567 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 52f896763048436482043ee9a516226d: loading cluster ID for follower catalog manager: success
I20260619 02:44:04.536201 28567 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 52f896763048436482043ee9a516226d: acquiring CA information for follower catalog manager: success
I20260619 02:44:04.536453 28567 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 52f896763048436482043ee9a516226d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 02:44:04.623340 28639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:04.620724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60610 (local address 127.5.241.125:35077)
0619 02:44:04.620850 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:04.620852 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:04.620862 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:44:04.620948 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:04.620949 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:04.621000 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:04.621079 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:44:04.621221 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:04.621731 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:04.622491 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:04.622696 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:04.622800 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:04.622973 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:44:04.623081 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:44:04.623164 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:04.623215 (+ 51us) server_negotiation.cc:299] Negotiation successful
0619 02:44:04.623236 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":59,"threads_started":1}
I20260619 02:44:04.627571 28645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:04.625133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38978 (local address 127.5.241.124:41559)
0619 02:44:04.625237 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:04.625239 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:04.625249 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:44:04.625340 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:04.625341 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:04.625383 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:04.625465 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:44:04.625556 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:04.626027 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:04.626742 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:04.626976 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:04.627048 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:04.627217 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:44:04.627326 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:44:04.627401 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:04.627452 (+ 51us) server_negotiation.cc:299] Negotiation successful
0619 02:44:04.627471 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81,"thread_start_us":48,"threads_started":1}
I20260619 02:44:04.632434 28645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:04.629227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38984 (local address 127.5.241.124:41559)
0619 02:44:04.629265 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:04.629267 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:04.629425 (+ 158us) server_negotiation.cc:407] Connection header received
0619 02:44:04.629509 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:04.629510 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:04.629519 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:04.629575 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:44:04.629741 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:04.630451 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:04.631623 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:04.631903 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:04.631958 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:04.632141 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:44:04.632172 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:44:04.632227 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:04.632339 (+ 112us) server_negotiation.cc:299] Negotiation successful
0619 02:44:04.632360 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:44:04.632434 28639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:04.629078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60612 (local address 127.5.241.125:35077)
0619 02:44:04.629220 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:04.629222 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:04.629359 (+ 137us) server_negotiation.cc:407] Connection header received
0619 02:44:04.629462 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:04.629463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:04.629472 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:04.629502 (+ 30us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:44:04.629669 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:04.630429 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:04.631580 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:04.631878 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:04.631956 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:04.632152 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:44:04.632181 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:44:04.632233 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:04.632332 (+ 99us) server_negotiation.cc:299] Negotiation successful
0619 02:44:04.632358 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121}
I20260619 02:44:04.632457 28651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:04.629079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34764 (local address 127.5.241.126:38479)
0619 02:44:04.629312 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:04.629314 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:04.629393 (+ 79us) server_negotiation.cc:407] Connection header received
0619 02:44:04.629481 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:04.629482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:04.629540 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:04.629618 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:44:04.629780 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:04.630370 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:04.631656 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:04.631861 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:04.631947 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:04.632137 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:44:04.632244 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:44:04.632293 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:04.632346 (+ 53us) server_negotiation.cc:299] Negotiation successful
0619 02:44:04.632368 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":45,"threads_started":1}
I20260619 02:44:04.639286 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 28288
I20260619 02:44:04.644635 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 28363
I20260619 02:44:04.649482 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 28438
2026-06-19T02:44:04Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2809 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:44:04 dist-test-slave-zpbg krb5kdc[28656](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:44:04 dist-test-slave-zpbg krb5kdc[28656](info): set up 2 sockets
Jun 19 02:44:04 dist-test-slave-zpbg krb5kdc[28656](info): commencing operation
krb5kdc: starting...
W20260619 02:44:05.691183 6085 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/krb5kdc/test-user.keytab.
Jun 19 02:44:05 dist-test-slave-zpbg krb5kdc[28656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837045, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:44:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:44:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.241.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
Entry for principal kudu/127.5.241.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab.
I20260619 02:44:05.809983 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:45905
--webserver_interface=127.5.241.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:37425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:45905,127.5.241.125:39683,127.5.241.124:33113
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:44:05.890342 28669 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 02:44:05.890573 28669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:44:05.890590 28669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:44:05.890611 28669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:44:05.891726 28669 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:44:05.891763 28669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:44:05.891777 28669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:44:05.891790 28669 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:44:05.891803 28669 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:44:05.893222 28669 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:37425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:45905,127.5.241.125:39683,127.5.241.124:33113
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.126.keytab
--principal=kudu/127.5.241.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.126:45905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.241.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:44:05.893409 28669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:44:05.893616 28669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:44:05.898470 28674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:05.898518 28677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:05.898653 28669 server_base.cc:1034] running on GCE node
W20260619 02:44:05.898470 28675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:05.898989 28669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:44:05.899341 28669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:44:05.900512 28669 hybrid_clock.cc:648] HybridClock initialized: now 1781837045900508 us; error 47 us; skew 500 ppm
Jun 19 02:44:05 dist-test-slave-zpbg krb5kdc[28656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837045, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:44:05.903247 28669 init.cc:377] Logged in from keytab as kudu/127.5.241.126@KRBTEST.COM (short username kudu)
I20260619 02:44:05.903728 28669 webserver.cc:460] Webserver started at http://127.5.241.126:45125/ using document root <none> and password file <none>
I20260619 02:44:05.903950 28669 fs_manager.cc:359] Metadata directory not provided
I20260619 02:44:05.904013 28669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:44:05.904150 28669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:44:05.905233 28669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/minicluster-data/master-0/data/instance:
uuid: "c37b3a825dd24050ab0b625340a2ffdd"
format_stamp: "Formatted at 2026-06-19 02:44:05 on dist-test-slave-zpbg"
server_key: "1777d4b62ab42002b71652ceb3fd5a61"
server_key_iv: "411607d06cffe1f1535a48f01fb8d87e"
server_key_version: "encryptionkey@0"
I20260619 02:44:05.905596 28669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/minicluster-data/master-0/wal/instance:
uuid: "c37b3a825dd24050ab0b625340a2ffdd"
format_stamp: "Formatted at 2026-06-19 02:44:05 on dist-test-slave-zpbg"
server_key: "1777d4b62ab42002b71652ceb3fd5a61"
server_key_iv: "411607d06cffe1f1535a48f01fb8d87e"
server_key_version: "encryptionkey@0"
I20260619 02:44:05.906948 28669 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:44:05.907698 28684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:05.907862 28669 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:44:05.907958 28669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/minicluster-data/master-0/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/minicluster-data/master-0/wal
uuid: "c37b3a825dd24050ab0b625340a2ffdd"
format_stamp: "Formatted at 2026-06-19 02:44:05 on dist-test-slave-zpbg"
server_key: "1777d4b62ab42002b71652ceb3fd5a61"
server_key_iv: "411607d06cffe1f1535a48f01fb8d87e"
server_key_version: "encryptionkey@0"
I20260619 02:44:05.908035 28669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:44:05.917647 28669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:44:05.919461 28669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:44:05.919611 28669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:44:05.923748 28669 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.126:45905
I20260619 02:44:05.923796 28736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.126:45905 every 8 connection(s)
I20260619 02:44:05.924158 28669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/minicluster-data/master-0/data/info.pb
I20260619 02:44:05.924592 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 28669
I20260619 02:44:05.924700 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/minicluster-data/master-0/wal/instance
I20260619 02:44:05.924938 6085 external_mini_cluster.cc:1428] Setting key 3d5dfe9c009e0a289d3c78e499d7704b
I20260619 02:44:05.925450 28737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:44:05.927503 28737 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45905 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.5.241.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.125@KRBTEST.COM" created.
Jun 19 02:44:05 dist-test-slave-zpbg krb5kdc[28656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837045, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:44:05.939297 28740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:05.928235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45905 (local address 127.0.0.1:34778)
0619 02:44:05.928408 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:05.928448 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:05.928941 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:05.929222 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:05.929224 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:05.929503 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:44:05.929644 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:05.929649 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:05.930674 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:05.930675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:05.931398 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:05.931401 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:05.931927 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:05.931928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:05.931994 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:05.932552 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:44:05.932560 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:44:05.935788 (+ 3228us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:44:05.937626 (+ 1838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:05.937631 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:05.937634 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:05.937828 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:05.938162 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:05.938162 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:05.938163 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:05.938257 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:05.938664 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:44:05.938668 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:44:05.938733 (+ 65us) client_negotiation.cc:769] Sending connection context
0619 02:44:05.938865 (+ 132us) client_negotiation.cc:240] Negotiation successful
0619 02:44:05.939117 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":89,"thread_start_us":82,"threads_started":1}
I20260619 02:44:05.939302 28741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:05.928264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34778 (local address 127.5.241.126:45905)
0619 02:44:05.928430 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:05.928440 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:05.928552 (+ 112us) server_negotiation.cc:407] Connection header received
0619 02:44:05.929096 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:05.929098 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:05.929118 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:05.929218 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:05.929769 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:05.930568 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:05.931500 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:05.931815 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:05.931941 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:05.935898 (+ 3957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:05.935905 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:05.935905 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:05.935912 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:05.937525 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:05.937981 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:05.937985 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:05.937985 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:05.938074 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:05.938323 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:05.938324 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:05.938324 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:05.938540 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:05.938684 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:05.938898 (+ 214us) server_negotiation.cc:299] Negotiation successful
0619 02:44:05.939039 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":93,"threads_started":1}
I20260619 02:44:05.940050 28737 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.241.125" port: 39683 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:44:05.940420 28740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:05.940306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.125:39683 (local address 127.0.0.1:53066)
0619 02:44:05.940358 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:05.940399 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.125:39683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260619 02:44:05.940586 28686 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.241.125:39683: connect: Connection refused (error 111)
I20260619 02:44:05.941035 28740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:05.940972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.125:39683 (local address 127.0.0.1:53074)
0619 02:44:05.940997 (+ 25us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:05.941019 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.125:39683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260619 02:44:05.941146 28737 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.241.125:39683: Network error: Client connection negotiation failed: client connection to 127.5.241.125:39683: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.125.keytab.
Entry for principal kudu/127.5.241.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.125.keytab.
I20260619 02:44:05.949856 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.125:39683
--webserver_interface=127.5.241.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.125.keytab
--principal=kudu/127.5.241.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:37425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:45905,127.5.241.125:39683,127.5.241.124:33113
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:44:05.990382 28737 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.241.125" port: 39683 } attempt: 1
I20260619 02:44:05.990934 28740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:05.990803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.125:39683 (local address 127.0.0.1:53082)
0619 02:44:05.990862 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:05.990904 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.125:39683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 02:44:05.991282 28740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:05.991204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.125:39683 (local address 127.0.0.1:53086)
0619 02:44:05.991242 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:05.991266 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.125:39683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260619 02:44:05.991376 28737 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.241.125:39683: Network error: Client connection negotiation failed: client connection to 127.5.241.125:39683: connect: Connection refused (error 111)
W20260619 02:44:06.030514 28744 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 02:44:06.030792 28744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:44:06.030824 28744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:44:06.030859 28744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:44:06.032006 28744 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:44:06.032047 28744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:44:06.032061 28744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:44:06.032074 28744 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:44:06.032094 28744 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:44:06.033517 28744 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:37425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:45905,127.5.241.125:39683,127.5.241.124:33113
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.125.keytab
--principal=kudu/127.5.241.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.125:39683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.241.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28744
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:44:06.033707 28744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:44:06.033876 28744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:44:06.038556 28744 server_base.cc:1034] running on GCE node
W20260619 02:44:06.038530 28750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:06.038529 28752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:06.038568 28749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:06.038951 28744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:44:06.039232 28744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:44:06.040364 28744 hybrid_clock.cc:648] HybridClock initialized: now 1781837046040356 us; error 28 us; skew 500 ppm
Jun 19 02:44:06 dist-test-slave-zpbg krb5kdc[28656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837046, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:44:06.042723 28744 init.cc:377] Logged in from keytab as kudu/127.5.241.125@KRBTEST.COM (short username kudu)
I20260619 02:44:06.043141 28744 webserver.cc:460] Webserver started at http://127.5.241.125:45307/ using document root <none> and password file <none>
I20260619 02:44:06.043371 28744 fs_manager.cc:359] Metadata directory not provided
I20260619 02:44:06.043435 28744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:44:06.043576 28744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:44:06.044602 28744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/minicluster-data/master-1/data/instance:
uuid: "282572e78c2a41bcbd79dd37da9df53b"
format_stamp: "Formatted at 2026-06-19 02:44:06 on dist-test-slave-zpbg"
server_key: "cdf4469ec2cf834e4e49b24b633741b4"
server_key_iv: "49b5fb4b09e5a4ca5890af32f62dc22f"
server_key_version: "encryptionkey@0"
I20260619 02:44:06.044961 28744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/minicluster-data/master-1/wal/instance:
uuid: "282572e78c2a41bcbd79dd37da9df53b"
format_stamp: "Formatted at 2026-06-19 02:44:06 on dist-test-slave-zpbg"
server_key: "cdf4469ec2cf834e4e49b24b633741b4"
server_key_iv: "49b5fb4b09e5a4ca5890af32f62dc22f"
server_key_version: "encryptionkey@0"
I20260619 02:44:06.046248 28744 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260619 02:44:06.046991 28759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:06.047154 28744 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:44:06.047238 28744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/minicluster-data/master-1/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/minicluster-data/master-1/wal
uuid: "282572e78c2a41bcbd79dd37da9df53b"
format_stamp: "Formatted at 2026-06-19 02:44:06 on dist-test-slave-zpbg"
server_key: "cdf4469ec2cf834e4e49b24b633741b4"
server_key_iv: "49b5fb4b09e5a4ca5890af32f62dc22f"
server_key_version: "encryptionkey@0"
I20260619 02:44:06.047312 28744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:44:06.059502 28737 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.241.125" port: 39683 } attempt: 2
I20260619 02:44:06.059917 28740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:06.059797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.125:39683 (local address 127.0.0.1:53094)
0619 02:44:06.059854 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:06.059889 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.125:39683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260619 02:44:06.060237 28740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:06.060151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.125:39683 (local address 127.0.0.1:53106)
0619 02:44:06.060191 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:06.060214 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.125:39683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260619 02:44:06.060323 28737 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.241.125:39683: Network error: Client connection negotiation failed: client connection to 127.5.241.125:39683: connect: Connection refused (error 111)
I20260619 02:44:06.069985 28744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:44:06.072021 28744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:44:06.072176 28744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:44:06.076053 28744 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.125:39683
I20260619 02:44:06.076090 28811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.125:39683 every 8 connection(s)
I20260619 02:44:06.076445 28744 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/minicluster-data/master-1/data/info.pb
I20260619 02:44:06.077378 28812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:44:06.078869 28812 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45905 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:44:06 dist-test-slave-zpbg krb5kdc[28656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837046, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.125@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:44:06.084869 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 28744
I20260619 02:44:06.085000 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/minicluster-data/master-1/wal/instance
I20260619 02:44:06.085222 6085 external_mini_cluster.cc:1428] Setting key e7de6cb4e8e5a96464639861491d6b9e
I20260619 02:44:06.087756 28741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:06.079522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34788 (local address 127.5.241.126:45905)
0619 02:44:06.079658 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:06.079660 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:06.079820 (+ 160us) server_negotiation.cc:407] Connection header received
0619 02:44:06.080314 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:06.080316 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:06.080328 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:06.080386 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:06.080989 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:06.081473 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:06.082461 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:06.082662 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:06.082746 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:06.084903 (+ 2157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:06.084905 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:06.084905 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:06.084913 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:06.086385 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:06.086816 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:06.086817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:06.086817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:06.086875 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:06.087124 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:06.087125 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:06.087125 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:06.087243 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:06.087316 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:06.087583 (+ 267us) server_negotiation.cc:299] Negotiation successful
0619 02:44:06.087663 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 02:44:06.087982 28814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:06.079452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45905 (local address 127.0.0.1:34788)
0619 02:44:06.079689 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:06.079733 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:06.080211 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:06.080457 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:06.080461 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:06.080703 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:44:06.080891 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:06.080894 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:06.081559 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:06.081560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:06.082361 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:06.082363 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:06.082794 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:06.082796 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:06.082916 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:06.083415 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:44:06.083424 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:44:06.084804 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:44:06.086486 (+ 1682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:06.086493 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:06.086495 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:06.086721 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:06.086946 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:06.086947 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:06.086948 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:06.087057 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:06.087308 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:44:06.087314 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:44:06.087450 (+ 136us) client_negotiation.cc:769] Sending connection context
0619 02:44:06.087562 (+ 112us) client_negotiation.cc:240] Negotiation successful
0619 02:44:06.087714 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":76,"threads_started":1}
I20260619 02:44:06.088496 28812 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.241.125" port: 39683 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.5.241.124@KRBTEST.COM; defaulting to no policy
Jun 19 02:44:06 dist-test-slave-zpbg krb5kdc[28656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837046, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.125@KRBTEST.COM for kudu/127.5.241.125@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.241.124@KRBTEST.COM" created.
I20260619 02:44:06.095544 28814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:06.088792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.125:39683 (local address 127.0.0.1:53110)
0619 02:44:06.088942 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:06.088955 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:06.089015 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:06.089158 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:06.089158 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:06.089290 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:44:06.089331 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:06.089333 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:06.090161 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:06.090162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:06.090653 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:06.090655 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:06.091167 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:06.091168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:06.091233 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:06.091632 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:44:06.091636 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:44:06.092742 (+ 1106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:44:06.094398 (+ 1656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:06.094398 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:06.094399 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:06.094560 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:06.094831 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:06.094832 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:06.094833 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:06.094906 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:06.095223 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:44:06.095224 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:44:06.095262 (+ 38us) client_negotiation.cc:769] Sending connection context
0619 02:44:06.095333 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 02:44:06.095429 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133}
I20260619 02:44:06.095810 28816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:06.088834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53110 (local address 127.5.241.125:39683)
0619 02:44:06.088980 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:06.088985 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:06.089017 (+ 32us) server_negotiation.cc:407] Connection header received
0619 02:44:06.089071 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:06.089074 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:06.089087 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:06.089157 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:06.089403 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:06.090063 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:06.090744 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:06.091082 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:06.091168 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:06.092836 (+ 1668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:06.092843 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:06.092843 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:06.092849 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:06.094298 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:06.094650 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:06.094653 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:06.094653 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:06.094706 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:06.094973 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:06.094974 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:06.094975 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:06.095160 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:06.095246 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:06.095362 (+ 116us) server_negotiation.cc:299] Negotiation successful
0619 02:44:06.095496 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":112,"thread_start_us":82,"threads_started":1}
I20260619 02:44:06.096410 28812 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.241.124" port: 33113 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:44:06.096742 28814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:06.096637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.124:33113 (local address 127.0.0.1:37674)
0619 02:44:06.096681 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:06.096720 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.124:33113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260619 02:44:06.096894 28760 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.241.124:33113: connect: Connection refused (error 111)
I20260619 02:44:06.097348 28814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:06.097222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.124:33113 (local address 127.0.0.1:37684)
0619 02:44:06.097290 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:06.097318 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.124:33113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260619 02:44:06.097492 28812 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.241.124:33113: Network error: Client connection negotiation failed: client connection to 127.5.241.124:33113: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.241.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.124.keytab.
Entry for principal kudu/127.5.241.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.124.keytab.
I20260619 02:44:06.110060 6085 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskflgrG6/build/release/bin/kudu
/tmp/dist-test-taskflgrG6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.124:33113
--webserver_interface=127.5.241.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.124.keytab
--principal=kudu/127.5.241.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.241.84:37425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.241.126:45905,127.5.241.125:39683,127.5.241.124:33113
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:44:06.146706 28812 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.241.124" port: 33113 } attempt: 1
I20260619 02:44:06.147220 28814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:06.147095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.124:33113 (local address 127.0.0.1:37692)
0619 02:44:06.147153 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:06.147193 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.124:33113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260619 02:44:06.147568 28814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:06.147473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.124:33113 (local address 127.0.0.1:37704)
0619 02:44:06.147516 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:06.147541 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.124:33113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260619 02:44:06.147658 28812 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.241.124:33113: Network error: Client connection negotiation failed: client connection to 127.5.241.124:33113: connect: Connection refused (error 111)
I20260619 02:44:06.151468 28737 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.241.125" port: 39683 } attempt: 3
Jun 19 02:44:06 dist-test-slave-zpbg krb5kdc[28656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837045, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for kudu/127.5.241.125@KRBTEST.COM
I20260619 02:44:06.158382 28740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:06.151758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.125:39683 (local address 127.0.0.1:53118)
0619 02:44:06.151812 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:06.151823 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:06.151869 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:06.152017 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:06.152018 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:06.152189 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:44:06.152253 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:06.152255 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:06.152898 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:06.152899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:06.153500 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:06.153502 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:06.153884 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:06.153885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:06.153965 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:06.154454 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:44:06.154458 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:44:06.155842 (+ 1384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:44:06.157495 (+ 1653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:06.157496 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:06.157496 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:06.157661 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:06.157850 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:06.157851 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:06.157851 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:06.157899 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:06.158155 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:44:06.158156 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:44:06.158194 (+ 38us) client_negotiation.cc:769] Sending connection context
0619 02:44:06.158244 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 02:44:06.158316 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260619 02:44:06.158447 28816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:06.151838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53118 (local address 127.5.241.125:39683)
0619 02:44:06.151899 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:06.151901 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:06.151911 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:44:06.151938 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:06.151939 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:06.151948 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:06.152012 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:06.152332 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:06.152821 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:06.153577 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:06.153802 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:06.153883 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:06.155935 (+ 2052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:06.155937 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:06.155937 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:06.155945 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:06.157416 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:06.157727 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:06.157728 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:06.157729 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:06.157789 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:06.157951 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:06.157952 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:06.157952 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:06.158088 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:06.158151 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:06.158281 (+ 130us) server_negotiation.cc:299] Negotiation successful
0619 02:44:06.158360 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 02:44:06.158850 28737 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.241.124" port: 33113 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:44:06.159183 28740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:06.159076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.124:33113 (local address 127.0.0.1:37720)
0619 02:44:06.159127 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:06.159160 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.124:33113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260619 02:44:06.159444 28740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:06.159363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.124:33113 (local address 127.0.0.1:37726)
0619 02:44:06.159403 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:06.159428 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.124:33113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260619 02:44:06.159519 28737 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.241.124:33113: Network error: Client connection negotiation failed: client connection to 127.5.241.124:33113: connect: Connection refused (error 111)
I20260619 02:44:06.190657 28737 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.241.124" port: 33113 } attempt: 1
I20260619 02:44:06.191183 28740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:06.191033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.124:33113 (local address 127.0.0.1:37740)
0619 02:44:06.191104 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:06.191147 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.124:33113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260619 02:44:06.191586 28740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:06.191473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.124:33113 (local address 127.0.0.1:37752)
0619 02:44:06.191526 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:06.191559 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.124:33113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260619 02:44:06.191700 28737 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.241.124:33113: Network error: Client connection negotiation failed: client connection to 127.5.241.124:33113: connect: Connection refused (error 111)
W20260619 02:44:06.191534 28819 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 02:44:06.191808 28819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:44:06.191843 28819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:44:06.191883 28819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:44:06.192996 28819 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:44:06.193056 28819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:44:06.193080 28819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:44:06.193111 28819 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:44:06.193137 28819 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:44:06.194615 28819 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.241.84:37425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.241.126:45905,127.5.241.125:39683,127.5.241.124:33113
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/krb5kdc/kudu_127.5.241.124.keytab
--principal=kudu/127.5.241.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.241.124:33113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.241.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:53:23 UTC on ccycloud.quasar-wqwdzy.root.comops.site
build id 4454976
I20260619 02:44:06.194892 28819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:44:06.195111 28819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:44:06.200131 28824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:06.200244 28819 server_base.cc:1034] running on GCE node
W20260619 02:44:06.200163 28825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:06.200323 28827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:06.200556 28819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:44:06.200840 28819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:44:06.201980 28819 hybrid_clock.cc:648] HybridClock initialized: now 1781837046201969 us; error 30 us; skew 500 ppm
Jun 19 02:44:06 dist-test-slave-zpbg krb5kdc[28656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837046, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:44:06.204877 28819 init.cc:377] Logged in from keytab as kudu/127.5.241.124@KRBTEST.COM (short username kudu)
I20260619 02:44:06.205368 28819 webserver.cc:460] Webserver started at http://127.5.241.124:35103/ using document root <none> and password file <none>
I20260619 02:44:06.205622 28819 fs_manager.cc:359] Metadata directory not provided
I20260619 02:44:06.205690 28819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:44:06.205840 28819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:44:06.206987 28819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/minicluster-data/master-2/data/instance:
uuid: "34f5d73fbcc1450c913e120b6bae542f"
format_stamp: "Formatted at 2026-06-19 02:44:06 on dist-test-slave-zpbg"
server_key: "4722824280125d1397ed24975329bb92"
server_key_iv: "5309d2f7f8cd26167ba91c785c2832bb"
server_key_version: "encryptionkey@0"
I20260619 02:44:06.207355 28819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/minicluster-data/master-2/wal/instance:
uuid: "34f5d73fbcc1450c913e120b6bae542f"
format_stamp: "Formatted at 2026-06-19 02:44:06 on dist-test-slave-zpbg"
server_key: "4722824280125d1397ed24975329bb92"
server_key_iv: "5309d2f7f8cd26167ba91c785c2832bb"
server_key_version: "encryptionkey@0"
I20260619 02:44:06.208666 28819 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:44:06.209405 28834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:06.209570 28819 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:44:06.209661 28819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/minicluster-data/master-2/data,/tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/minicluster-data/master-2/wal
uuid: "34f5d73fbcc1450c913e120b6bae542f"
format_stamp: "Formatted at 2026-06-19 02:44:06 on dist-test-slave-zpbg"
server_key: "4722824280125d1397ed24975329bb92"
server_key_iv: "5309d2f7f8cd26167ba91c785c2832bb"
server_key_version: "encryptionkey@0"
I20260619 02:44:06.209717 28819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:44:06.215781 28812 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.241.124" port: 33113 } attempt: 2
I20260619 02:44:06.216168 28814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:06.216054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.124:33113 (local address 127.0.0.1:37754)
0619 02:44:06.216106 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:06.216141 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.124:33113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 02:44:06.216480 28814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:06.216393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.124:33113 (local address 127.0.0.1:37768)
0619 02:44:06.216433 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:06.216462 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.241.124:33113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260619 02:44:06.216559 28812 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.241.124:33113: Network error: Client connection negotiation failed: client connection to 127.5.241.124:33113: connect: Connection refused (error 111)
I20260619 02:44:06.250597 28819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:44:06.252480 28819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:44:06.252647 28819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:44:06.256476 28819 rpc_server.cc:307] RPC server started. Bound to: 127.5.241.124:33113
I20260619 02:44:06.256517 28886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.241.124:33113 every 8 connection(s)
I20260619 02:44:06.256891 28819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/minicluster-data/master-2/data/info.pb
I20260619 02:44:06.257854 28887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:44:06.259434 28887 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45905 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:44:06.264855 6085 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskflgrG6/build/release/bin/kudu as pid 28819
Jun 19 02:44:06 dist-test-slave-zpbg krb5kdc[28656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837046, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.124@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:44:06.264977 6085 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskflgrG6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836772852275-6085-0/minicluster-data/master-2/wal/instance
I20260619 02:44:06.265196 6085 external_mini_cluster.cc:1428] Setting key 6d08a868aa387739bdc70ebd790391b8
I20260619 02:44:06.266810 28737 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.241.124" port: 33113 } attempt: 2
I20260619 02:44:06.268718 28741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:06.260096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34790 (local address 127.5.241.126:45905)
0619 02:44:06.260155 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:06.260157 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:06.260360 (+ 203us) server_negotiation.cc:407] Connection header received
0619 02:44:06.260845 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:06.260846 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:06.260857 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:06.260914 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:06.261511 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:06.261978 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:06.263023 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:06.263218 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:06.263299 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:06.265563 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:06.265564 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:06.265565 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:06.265573 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:06.267281 (+ 1708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:06.267698 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:06.267699 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:06.267700 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:06.267755 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:06.268021 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:06.268023 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:06.268023 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:06.268144 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:06.268214 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:06.268499 (+ 285us) server_negotiation.cc:299] Negotiation successful
0619 02:44:06.268621 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":32}
I20260619 02:44:06.268848 28889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:06.260071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.126:45905 (local address 127.0.0.1:34790)
0619 02:44:06.260245 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:06.260283 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:06.260746 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:06.260988 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:06.260991 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:06.261235 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:44:06.261421 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:06.261425 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:06.262084 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:06.262085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:06.262934 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:06.262936 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:06.263318 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:06.263319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:06.263443 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:06.263889 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:44:06.263904 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:44:06.265449 (+ 1545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:44:06.267352 (+ 1903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:06.267356 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:06.267358 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:06.267638 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:06.267817 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:06.267818 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:06.267818 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:06.267954 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:06.268216 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:44:06.268223 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:44:06.268350 (+ 127us) client_negotiation.cc:769] Sending connection context
0619 02:44:06.268456 (+ 106us) client_negotiation.cc:240] Negotiation successful
0619 02:44:06.268610 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":79,"threads_started":1}
I20260619 02:44:06.269377 28887 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.241.125" port: 39683 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:44:06 dist-test-slave-zpbg krb5kdc[28656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837045, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.241.124@KRBTEST.COM
Jun 19 02:44:06 dist-test-slave-zpbg krb5kdc[28656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837045, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.126@KRBTEST.COM for kudu/127.5.241.124@KRBTEST.COM
Jun 19 02:44:06 dist-test-slave-zpbg krb5kdc[28656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837046, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.124@KRBTEST.COM for kudu/127.5.241.125@KRBTEST.COM
I20260619 02:44:06.275781 28891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:06.265658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37784 (local address 127.5.241.124:33113)
0619 02:44:06.265792 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:06.265796 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:06.265814 (+ 18us) server_negotiation.cc:407] Connection header received
0619 02:44:06.265838 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:06.265840 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:06.265854 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:06.265933 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:06.266723 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:06.267829 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:06.268825 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:06.269323 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:06.269395 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:06.271988 (+ 2593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:06.271996 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:06.271996 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:06.272002 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:06.274402 (+ 2400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:06.274830 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:06.274833 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:06.274834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:06.274889 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:06.275120 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:06.275121 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:06.275121 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:06.275282 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:06.275477 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:06.275568 (+ 91us) server_negotiation.cc:299] Negotiation successful
0619 02:44:06.275651 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":82,"threads_started":1}
I20260619 02:44:06.276443 28740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:06.267044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.124:33113 (local address 127.0.0.1:37792)
0619 02:44:06.267103 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:06.267112 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:06.267157 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:06.267397 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:06.267398 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:06.267519 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:44:06.267551 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:06.267553 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:06.268443 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:06.268445 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:06.269745 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:06.269747 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:06.270076 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:06.270077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:06.270137 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:06.270461 (+ 324us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:44:06.270467 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:44:06.272410 (+ 1943us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:44:06.274911 (+ 2501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:06.274911 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:06.274912 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:06.275133 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:06.275560 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:06.275560 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:06.275561 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:06.275639 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:06.276157 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:44:06.276158 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:44:06.276207 (+ 49us) client_negotiation.cc:769] Sending connection context
0619 02:44:06.276259 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 02:44:06.276346 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260619 02:44:06.276818 28892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:06.267117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37792 (local address 127.5.241.124:33113)
0619 02:44:06.267263 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:06.267265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:06.267272 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:44:06.267291 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:06.267292 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:06.267302 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:06.267392 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:06.267630 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:06.268366 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:06.269825 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:06.270005 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:06.270053 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:06.272504 (+ 2451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:06.272505 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:06.272505 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:06.272511 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:06.274827 (+ 2316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:06.275201 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:06.275202 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:06.275202 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:06.275258 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:06.275716 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:06.275717 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:06.275717 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:06.276068 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:06.276552 (+ 484us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:06.276605 (+ 53us) server_negotiation.cc:299] Negotiation successful
0619 02:44:06.276681 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":72,"threads_started":1}
I20260619 02:44:06.277796 28889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:06.269597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.125:39683 (local address 127.0.0.1:53124)
0619 02:44:06.270830 (+ 1233us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:06.270840 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:06.270901 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:06.271035 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:06.271035 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:06.271178 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:44:06.271217 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:06.271218 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:06.271899 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:06.271900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:06.272649 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:06.272652 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:06.273112 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:06.273113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:06.273196 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:06.273523 (+ 327us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:44:06.273527 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:44:06.275006 (+ 1479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:44:06.276799 (+ 1793us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:06.276800 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:06.276801 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:06.276983 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:06.277186 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:06.277187 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:06.277188 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:06.277242 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:06.277509 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:44:06.277509 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:44:06.277548 (+ 39us) client_negotiation.cc:769] Sending connection context
0619 02:44:06.277628 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 02:44:06.277707 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1218}
I20260619 02:44:06.277866 28816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:06.269650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53124 (local address 127.5.241.125:39683)
0619 02:44:06.269693 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:06.269694 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:06.270894 (+ 1200us) server_negotiation.cc:407] Connection header received
0619 02:44:06.270963 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:06.270964 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:06.270976 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:06.271033 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:06.271301 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:06.271816 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:06.272738 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:06.273029 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:06.273109 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:06.275090 (+ 1981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:06.275091 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:06.275092 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:06.275098 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:06.276718 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:06.277058 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:06.277059 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:06.277059 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:06.277115 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:06.277302 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:06.277303 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:06.277304 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:06.277443 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:06.277510 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:06.277675 (+ 165us) server_negotiation.cc:299] Negotiation successful
0619 02:44:06.277752 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 02:44:06.278158 28887 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.241.124" port: 33113 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:44:06.278674 28737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c37b3a825dd24050ab0b625340a2ffdd: Bootstrap starting.
I20260619 02:44:06.279285 28737 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c37b3a825dd24050ab0b625340a2ffdd: Neither blocks nor log segments found. Creating new log.
I20260619 02:44:06.279610 28737 log.cc:824] T 00000000000000000000000000000000 P c37b3a825dd24050ab0b625340a2ffdd: Log is configured to *not* fsync() on all Append() calls
I20260619 02:44:06.280602 28737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c37b3a825dd24050ab0b625340a2ffdd: No bootstrap required, opened a new log
Jun 19 02:44:06 dist-test-slave-zpbg krb5kdc[28656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837045, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.241.126@KRBTEST.COM
I20260619 02:44:06.282398 28737 raft_consensus.cc:348] T 00000000000000000000000000000000 P c37b3a825dd24050ab0b625340a2ffdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c37b3a825dd24050ab0b625340a2ffdd" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45905 } } peers { permanent_uuid: "282572e78c2a41bcbd79dd37da9df53b" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 39683 } } peers { permanent_uuid: "34f5d73fbcc1450c913e120b6bae542f" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 33113 } }
I20260619 02:44:06.282509 28737 raft_consensus.cc:729] T 00000000000000000000000000000000 P c37b3a825dd24050ab0b625340a2ffdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c37b3a825dd24050ab0b625340a2ffdd, State: Initialized, Role: FOLLOWER
I20260619 02:44:06.282581 28737 consensus_queue.cc:260] T 00000000000000000000000000000000 P c37b3a825dd24050ab0b625340a2ffdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c37b3a825dd24050ab0b625340a2ffdd" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45905 } } peers { permanent_uuid: "282572e78c2a41bcbd79dd37da9df53b" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 39683 } } peers { permanent_uuid: "34f5d73fbcc1450c913e120b6bae542f" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 33113 } }
I20260619 02:44:06.282924 28737 sys_catalog.cc:564] T 00000000000000000000000000000000 P c37b3a825dd24050ab0b625340a2ffdd [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:44:06.282912 28894 sys_catalog.cc:455] T 00000000000000000000000000000000 P c37b3a825dd24050ab0b625340a2ffdd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c37b3a825dd24050ab0b625340a2ffdd" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45905 } } peers { permanent_uuid: "282572e78c2a41bcbd79dd37da9df53b" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 39683 } } peers { permanent_uuid: "34f5d73fbcc1450c913e120b6bae542f" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 33113 } } }
I20260619 02:44:06.283170 28894 sys_catalog.cc:458] T 00000000000000000000000000000000 P c37b3a825dd24050ab0b625340a2ffdd [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:44:06.284228 28741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:06.276489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34802 (local address 127.5.241.126:45905)
0619 02:44:06.276590 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:06.276592 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:06.276761 (+ 169us) server_negotiation.cc:407] Connection header received
0619 02:44:06.276830 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:06.276831 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:06.276840 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:06.276887 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:06.277490 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:06.278171 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:06.278863 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:06.279060 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:06.279134 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:06.281262 (+ 2128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:06.281263 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:06.281264 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:06.281271 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:06.282865 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:06.283335 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:06.283336 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:06.283336 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:06.283399 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:06.283658 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:06.283659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:06.283659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:06.283799 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:06.283998 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:06.284041 (+ 43us) server_negotiation.cc:299] Negotiation successful
0619 02:44:06.284122 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
W20260619 02:44:06.285074 28905 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c37b3a825dd24050ab0b625340a2ffdd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:44:06.285140 28905 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Jun 19 02:44:06 dist-test-slave-zpbg krb5kdc[28656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837046, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.124@KRBTEST.COM for kudu/127.5.241.124@KRBTEST.COM
I20260619 02:44:06.286382 28669 master_runner.cc:186] Error getting master registration for 127.5.241.125:39683: OK, instance_id { permanent_uuid: "282572e78c2a41bcbd79dd37da9df53b" instance_seqno: 1781837046074971 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 02:44:06.290069 28669 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Jun 19 02:44:06 dist-test-slave-zpbg krb5kdc[28656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837045, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.241.125@KRBTEST.COM
I20260619 02:44:06.290416 28892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:06.280008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37794 (local address 127.5.241.124:33113)
0619 02:44:06.280045 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:06.280047 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:06.283078 (+ 3031us) server_negotiation.cc:407] Connection header received
0619 02:44:06.283096 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:06.283097 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:06.283105 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:06.283433 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:06.283458 (+ 25us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:06.284204 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:06.285131 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:06.285447 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:06.285523 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:06.287461 (+ 1938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:06.287462 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:06.287463 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:06.287482 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:06.289174 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:06.289625 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:06.289626 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:06.289627 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:06.289668 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:06.289854 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:06.289854 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:06.289855 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:06.290028 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:06.290071 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:06.290256 (+ 185us) server_negotiation.cc:299] Negotiation successful
0619 02:44:06.290326 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 02:44:06.290699 28889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:06.279947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.124:33113 (local address 127.0.0.1:37794)
0619 02:44:06.282963 (+ 3016us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:06.282976 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:06.283029 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:06.283186 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:06.283187 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:06.283330 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:44:06.283368 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:06.283370 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:06.284297 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:06.284298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:06.285047 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:06.285049 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:06.285532 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:06.285533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:06.285579 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:06.285969 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:44:06.285974 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:44:06.287376 (+ 1402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:44:06.289291 (+ 1915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:06.289291 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:06.289292 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:06.289549 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:06.289731 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:06.289732 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:06.289732 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:06.289787 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:06.290113 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:44:06.290113 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:44:06.290154 (+ 41us) client_negotiation.cc:769] Sending connection context
0619 02:44:06.290529 (+ 375us) client_negotiation.cc:240] Negotiation successful
0619 02:44:06.290622 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2987}
I20260619 02:44:06.291940 28887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34f5d73fbcc1450c913e120b6bae542f: Bootstrap starting.
I20260619 02:44:06.292539 28887 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34f5d73fbcc1450c913e120b6bae542f: Neither blocks nor log segments found. Creating new log.
I20260619 02:44:06.292806 28887 log.cc:824] T 00000000000000000000000000000000 P 34f5d73fbcc1450c913e120b6bae542f: Log is configured to *not* fsync() on all Append() calls
I20260619 02:44:06.293601 28887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34f5d73fbcc1450c913e120b6bae542f: No bootstrap required, opened a new log
I20260619 02:44:06.293936 28816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:06.285044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53128 (local address 127.5.241.125:39683)
0619 02:44:06.285108 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:06.285110 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:06.285119 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:44:06.285141 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:06.285142 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:06.285152 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:06.285200 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:06.285862 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:06.286569 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:06.287606 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:06.287899 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:06.287962 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:06.291056 (+ 3094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:06.291057 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:06.291057 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:06.291064 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:06.292754 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:06.293130 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:06.293132 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:06.293132 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:06.293194 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:06.293405 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:06.293406 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:06.293406 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:06.293546 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:06.293733 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:06.293775 (+ 42us) server_negotiation.cc:299] Negotiation successful
0619 02:44:06.293847 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 02:44:06.295367 6085 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:44:06.295809 28887 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34f5d73fbcc1450c913e120b6bae542f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c37b3a825dd24050ab0b625340a2ffdd" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45905 } } peers { permanent_uuid: "282572e78c2a41bcbd79dd37da9df53b" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 39683 } } peers { permanent_uuid: "34f5d73fbcc1450c913e120b6bae542f" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 33113 } }
I20260619 02:44:06.295949 28887 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34f5d73fbcc1450c913e120b6bae542f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34f5d73fbcc1450c913e120b6bae542f, State: Initialized, Role: FOLLOWER
I20260619 02:44:06.296041 28887 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34f5d73fbcc1450c913e120b6bae542f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c37b3a825dd24050ab0b625340a2ffdd" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45905 } } peers { permanent_uuid: "282572e78c2a41bcbd79dd37da9df53b" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 39683 } } peers { permanent_uuid: "34f5d73fbcc1450c913e120b6bae542f" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 33113 } }
I20260619 02:44:06.296221 28711 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "c37b3a825dd24050ab0b625340a2ffdd"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34802
I20260619 02:44:06.296327 28711 raft_consensus.cc:482] T 00000000000000000000000000000000 P c37b3a825dd24050ab0b625340a2ffdd [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 02:44:06.296391 28711 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c37b3a825dd24050ab0b625340a2ffdd [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:44:06.297061 28711 raft_consensus.cc:504] T 00000000000000000000000000000000 P c37b3a825dd24050ab0b625340a2ffdd [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c37b3a825dd24050ab0b625340a2ffdd" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45905 } } peers { permanent_uuid: "282572e78c2a41bcbd79dd37da9df53b" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 39683 } } peers { permanent_uuid: "34f5d73fbcc1450c913e120b6bae542f" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 33113 } }
I20260619 02:44:06.297290 28711 leader_election.cc:288] T 00000000000000000000000000000000 P c37b3a825dd24050ab0b625340a2ffdd [CANDIDATE]: Term 1 election: Requested vote from peers 282572e78c2a41bcbd79dd37da9df53b (127.5.241.125:39683), 34f5d73fbcc1450c913e120b6bae542f (127.5.241.124:33113)
I20260619 02:44:06.297545 28908 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34f5d73fbcc1450c913e120b6bae542f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c37b3a825dd24050ab0b625340a2ffdd" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45905 } } peers { permanent_uuid: "282572e78c2a41bcbd79dd37da9df53b" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 39683 } } peers { permanent_uuid: "34f5d73fbcc1450c913e120b6bae542f" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 33113 } } }
I20260619 02:44:06.297608 28887 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34f5d73fbcc1450c913e120b6bae542f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:44:06.297637 28908 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34f5d73fbcc1450c913e120b6bae542f [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:44:06.297668 28861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c37b3a825dd24050ab0b625340a2ffdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "34f5d73fbcc1450c913e120b6bae542f"
I20260619 02:44:06.297787 28861 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34f5d73fbcc1450c913e120b6bae542f [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:44:06.298602 28861 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 34f5d73fbcc1450c913e120b6bae542f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c37b3a825dd24050ab0b625340a2ffdd in term 1.
I20260619 02:44:06.298887 28685 leader_election.cc:302] T 00000000000000000000000000000000 P c37b3a825dd24050ab0b625340a2ffdd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f5d73fbcc1450c913e120b6bae542f, c37b3a825dd24050ab0b625340a2ffdd; no voters:
I20260619 02:44:06.299021 28894 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c37b3a825dd24050ab0b625340a2ffdd [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:44:06.299163 28894 raft_consensus.cc:686] T 00000000000000000000000000000000 P c37b3a825dd24050ab0b625340a2ffdd [term 1 LEADER]: Becoming Leader. State: Replica: c37b3a825dd24050ab0b625340a2ffdd, State: Running, Role: LEADER
I20260619 02:44:06.299249 28894 consensus_queue.cc:237] T 00000000000000000000000000000000 P c37b3a825dd24050ab0b625340a2ffdd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c37b3a825dd24050ab0b625340a2ffdd" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45905 } } peers { permanent_uuid: "282572e78c2a41bcbd79dd37da9df53b" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 39683 } } peers { permanent_uuid: "34f5d73fbcc1450c913e120b6bae542f" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 33113 } }
W20260619 02:44:06.299458 28919 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 34f5d73fbcc1450c913e120b6bae542f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:44:06.299683 28919 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:44:06.299695 28894 sys_catalog.cc:455] T 00000000000000000000000000000000 P c37b3a825dd24050ab0b625340a2ffdd [sys.catalog]: SysCatalogTable state changed. Reason: New leader c37b3a825dd24050ab0b625340a2ffdd. Latest consensus state: current_term: 1 leader_uuid: "c37b3a825dd24050ab0b625340a2ffdd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c37b3a825dd24050ab0b625340a2ffdd" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45905 } } peers { permanent_uuid: "282572e78c2a41bcbd79dd37da9df53b" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 39683 } } peers { permanent_uuid: "34f5d73fbcc1450c913e120b6bae542f" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 33113 } } }
I20260619 02:44:06.299767 28894 sys_catalog.cc:458] T 00000000000000000000000000000000 P c37b3a825dd24050ab0b625340a2ffdd [sys.catalog]: This master's current role is: LEADER
I20260619 02:44:06.300802 28927 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:44:06.301049 28927 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:44:06.301602 28819 master_runner.cc:186] Error getting master registration for 127.5.241.125:39683: OK, instance_id { permanent_uuid: "282572e78c2a41bcbd79dd37da9df53b" instance_seqno: 1781837046074971 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 02:44:06.303759 28819 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 02:44:06.306016 28861 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 34f5d73fbcc1450c913e120b6bae542f [term 1 FOLLOWER]: Refusing update from remote peer c37b3a825dd24050ab0b625340a2ffdd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:44:06.306321 28894 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c37b3a825dd24050ab0b625340a2ffdd [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f5d73fbcc1450c913e120b6bae542f" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 33113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:44:06.307672 28812 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.241.124" port: 33113 } attempt: 3
I20260619 02:44:06.308321 28908 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34f5d73fbcc1450c913e120b6bae542f [sys.catalog]: SysCatalogTable state changed. Reason: New leader c37b3a825dd24050ab0b625340a2ffdd. Latest consensus state: current_term: 1 leader_uuid: "c37b3a825dd24050ab0b625340a2ffdd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c37b3a825dd24050ab0b625340a2ffdd" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45905 } } peers { permanent_uuid: "282572e78c2a41bcbd79dd37da9df53b" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 39683 } } peers { permanent_uuid: "34f5d73fbcc1450c913e120b6bae542f" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 33113 } } }
I20260619 02:44:06.308454 28908 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34f5d73fbcc1450c913e120b6bae542f [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:44:06.312044 28816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:06.300549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53130 (local address 127.5.241.125:39683)
0619 02:44:06.300585 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:06.300587 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:06.301528 (+ 941us) server_negotiation.cc:407] Connection header received
0619 02:44:06.301642 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:06.301643 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:06.301653 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:06.301693 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:06.302874 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:06.303615 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:06.304831 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:06.305122 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:06.305204 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:06.307189 (+ 1985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:06.307190 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:06.307190 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:06.307197 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:06.310109 (+ 2912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:06.310500 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:06.310501 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:06.310502 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:06.310586 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:06.310796 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:06.310797 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:06.310798 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:06.310929 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:06.311122 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:06.311441 (+ 319us) server_negotiation.cc:299] Negotiation successful
0619 02:44:06.311954 (+ 513us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
Jun 19 02:44:06 dist-test-slave-zpbg krb5kdc[28656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837046, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.241.125@KRBTEST.COM for kudu/127.5.241.124@KRBTEST.COM
I20260619 02:44:06.316140 28814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:06.307874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.241.124:33113 (local address 127.0.0.1:37802)
0619 02:44:06.307946 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:06.307955 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:06.307988 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:06.308246 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:06.308248 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:06.308504 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:44:06.308546 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:06.308548 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:06.309439 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:06.309440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:06.310249 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:06.310252 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:06.310808 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:06.310809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:06.310865 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:06.311264 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:44:06.311269 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:44:06.312830 (+ 1561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:44:06.315090 (+ 2260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:06.315091 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:06.315091 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:06.315307 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:06.315503 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:06.315504 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:06.315504 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:06.315558 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:06.315840 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:44:06.315840 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:44:06.315883 (+ 43us) client_negotiation.cc:769] Sending connection context
0619 02:44:06.315944 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 02:44:06.316042 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260619 02:44:06.316510 28891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:06.308055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37802 (local address 127.5.241.124:33113)
0619 02:44:06.308096 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:06.308098 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:06.308107 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:44:06.308126 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:06.308126 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:06.308136 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:06.308196 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:06.308628 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:06.309368 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:06.310341 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:06.310683 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:06.310739 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:06.312905 (+ 2166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:06.312923 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:06.312924 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:06.312931 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:06.315005 (+ 2074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:06.315369 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:06.315370 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:06.315371 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:06.315430 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:06.315627 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:06.315628 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:06.315628 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:06.315757 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:06.315814 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:06.316329 (+ 515us) server_negotiation.cc:299] Negotiation successful
0619 02:44:06.316409 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 02:44:06.317997 28741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:06.300186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34814 (local address 127.5.241.126:45905)
0619 02:44:06.300228 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:06.300230 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:06.301340 (+ 1110us) server_negotiation.cc:407] Connection header received
0619 02:44:06.301429 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:06.301430 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:06.301440 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:06.301507 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:06.303563 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:06.304267 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:06.308636 (+ 4369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:06.308933 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:06.308993 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:06.313306 (+ 4313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:06.313308 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:06.313308 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:06.313316 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:06.315125 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:06.317075 (+ 1950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:06.317077 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:06.317077 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:06.317134 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:06.317343 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:06.317344 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:06.317345 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:06.317488 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:06.317582 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:06.317791 (+ 209us) server_negotiation.cc:299] Negotiation successful
0619 02:44:06.317877 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:44:06.318013 28812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 282572e78c2a41bcbd79dd37da9df53b: Bootstrap starting.
I20260619 02:44:06.318791 28812 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 282572e78c2a41bcbd79dd37da9df53b: Neither blocks nor log segments found. Creating new log.
I20260619 02:44:06.319087 28812 log.cc:824] T 00000000000000000000000000000000 P 282572e78c2a41bcbd79dd37da9df53b: Log is configured to *not* fsync() on all Append() calls
I20260619 02:44:06.320602 28812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 282572e78c2a41bcbd79dd37da9df53b: No bootstrap required, opened a new log
I20260619 02:44:06.321822 28892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:06.301284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37800 (local address 127.5.241.124:33113)
0619 02:44:06.301320 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:06.301321 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:06.311747 (+ 10426us) server_negotiation.cc:407] Connection header received
0619 02:44:06.311836 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:06.311837 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:06.311846 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:06.311874 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:06.312440 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:06.313476 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:06.315866 (+ 2390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:06.316197 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:06.316267 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:06.318293 (+ 2026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:06.318294 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:06.318294 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:06.318302 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:06.320081 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:06.320673 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:06.320675 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:06.320675 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:06.320730 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:06.321115 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:06.321116 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:06.321116 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:06.321265 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:06.321333 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:06.321653 (+ 320us) server_negotiation.cc:299] Negotiation successful
0619 02:44:06.321729 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 02:44:06.322620 28812 raft_consensus.cc:348] T 00000000000000000000000000000000 P 282572e78c2a41bcbd79dd37da9df53b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c37b3a825dd24050ab0b625340a2ffdd" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45905 } } peers { permanent_uuid: "282572e78c2a41bcbd79dd37da9df53b" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 39683 } } peers { permanent_uuid: "34f5d73fbcc1450c913e120b6bae542f" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 33113 } }
I20260619 02:44:06.322751 28812 raft_consensus.cc:729] T 00000000000000000000000000000000 P 282572e78c2a41bcbd79dd37da9df53b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 282572e78c2a41bcbd79dd37da9df53b, State: Initialized, Role: FOLLOWER
I20260619 02:44:06.322844 28812 consensus_queue.cc:260] T 00000000000000000000000000000000 P 282572e78c2a41bcbd79dd37da9df53b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c37b3a825dd24050ab0b625340a2ffdd" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45905 } } peers { permanent_uuid: "282572e78c2a41bcbd79dd37da9df53b" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 39683 } } peers { permanent_uuid: "34f5d73fbcc1450c913e120b6bae542f" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 33113 } }
I20260619 02:44:06.323155 28812 sys_catalog.cc:564] T 00000000000000000000000000000000 P 282572e78c2a41bcbd79dd37da9df53b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:44:06.323228 28785 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 282572e78c2a41bcbd79dd37da9df53b [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:44:06.324139 28785 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 282572e78c2a41bcbd79dd37da9df53b [term 1 FOLLOWER]: Refusing update from remote peer c37b3a825dd24050ab0b625340a2ffdd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:44:06.324437 28786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c37b3a825dd24050ab0b625340a2ffdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "282572e78c2a41bcbd79dd37da9df53b"
I20260619 02:44:06.325239 28786 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 282572e78c2a41bcbd79dd37da9df53b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c37b3a825dd24050ab0b625340a2ffdd in term 1.
I20260619 02:44:06.325733 28894 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c37b3a825dd24050ab0b625340a2ffdd [LEADER]: Connected to new peer: Peer: permanent_uuid: "282572e78c2a41bcbd79dd37da9df53b" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 39683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:44:06.326084 28894 sys_catalog.cc:455] T 00000000000000000000000000000000 P c37b3a825dd24050ab0b625340a2ffdd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c37b3a825dd24050ab0b625340a2ffdd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c37b3a825dd24050ab0b625340a2ffdd" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45905 } } peers { permanent_uuid: "282572e78c2a41bcbd79dd37da9df53b" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 39683 } } peers { permanent_uuid: "34f5d73fbcc1450c913e120b6bae542f" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 33113 } } }
I20260619 02:44:06.326162 28894 sys_catalog.cc:458] T 00000000000000000000000000000000 P c37b3a825dd24050ab0b625340a2ffdd [sys.catalog]: This master's current role is: LEADER
I20260619 02:44:06.326496 28927 catalog_manager.cc:1347] Generated new cluster ID: bfc14547cf3f4eb1b8b5012d8ea61dab
I20260619 02:44:06.326542 28927 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:44:06.328270 28931 mvcc.cc:205] Tried to move back new op lower bound from 7298404541665554432 to 7298404541642588160. Current Snapshot: MvccSnapshot[applied={T|T < 7298404541665554432 or (T in {7298404541665554432})}]
I20260619 02:44:06.328675 28949 sys_catalog.cc:455] T 00000000000000000000000000000000 P 282572e78c2a41bcbd79dd37da9df53b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c37b3a825dd24050ab0b625340a2ffdd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c37b3a825dd24050ab0b625340a2ffdd" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45905 } } peers { permanent_uuid: "282572e78c2a41bcbd79dd37da9df53b" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 39683 } } peers { permanent_uuid: "34f5d73fbcc1450c913e120b6bae542f" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 33113 } } }
I20260619 02:44:06.328864 28949 sys_catalog.cc:458] T 00000000000000000000000000000000 P 282572e78c2a41bcbd79dd37da9df53b [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:44:06.328975 28949 sys_catalog.cc:455] T 00000000000000000000000000000000 P 282572e78c2a41bcbd79dd37da9df53b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c37b3a825dd24050ab0b625340a2ffdd. Latest consensus state: current_term: 1 leader_uuid: "c37b3a825dd24050ab0b625340a2ffdd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c37b3a825dd24050ab0b625340a2ffdd" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45905 } } peers { permanent_uuid: "282572e78c2a41bcbd79dd37da9df53b" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 39683 } } peers { permanent_uuid: "34f5d73fbcc1450c913e120b6bae542f" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 33113 } } }
I20260619 02:44:06.329082 28949 sys_catalog.cc:458] T 00000000000000000000000000000000 P 282572e78c2a41bcbd79dd37da9df53b [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:44:06.329175 28949 sys_catalog.cc:455] T 00000000000000000000000000000000 P 282572e78c2a41bcbd79dd37da9df53b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c37b3a825dd24050ab0b625340a2ffdd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c37b3a825dd24050ab0b625340a2ffdd" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45905 } } peers { permanent_uuid: "282572e78c2a41bcbd79dd37da9df53b" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 39683 } } peers { permanent_uuid: "34f5d73fbcc1450c913e120b6bae542f" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 33113 } } }
I20260619 02:44:06.329274 28949 sys_catalog.cc:458] T 00000000000000000000000000000000 P 282572e78c2a41bcbd79dd37da9df53b [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:44:06.330813 28908 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34f5d73fbcc1450c913e120b6bae542f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c37b3a825dd24050ab0b625340a2ffdd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c37b3a825dd24050ab0b625340a2ffdd" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45905 } } peers { permanent_uuid: "282572e78c2a41bcbd79dd37da9df53b" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 39683 } } peers { permanent_uuid: "34f5d73fbcc1450c913e120b6bae542f" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 33113 } } }
I20260619 02:44:06.330906 28908 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34f5d73fbcc1450c913e120b6bae542f [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:44:06.330896 28950 mvcc.cc:205] Tried to move back new op lower bound from 7298404541665554432 to 7298404541642588160. Current Snapshot: MvccSnapshot[applied={T|T < 7298404541665554432}]
W20260619 02:44:06.331713 28953 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 282572e78c2a41bcbd79dd37da9df53b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:44:06.331804 28953 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:44:06.336378 28932 mvcc.cc:205] Tried to move back new op lower bound from 7298404541665554432 to 7298404541642588160. Current Snapshot: MvccSnapshot[applied={T|T < 7298404541665554432 or (T in {7298404541665554432})}]
I20260619 02:44:06.336653 28920 sys_catalog.cc:455] T 00000000000000000000000000000000 P c37b3a825dd24050ab0b625340a2ffdd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c37b3a825dd24050ab0b625340a2ffdd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c37b3a825dd24050ab0b625340a2ffdd" member_type: VOTER last_known_addr { host: "127.5.241.126" port: 45905 } } peers { permanent_uuid: "282572e78c2a41bcbd79dd37da9df53b" member_type: VOTER last_known_addr { host: "127.5.241.125" port: 39683 } } peers { permanent_uuid: "34f5d73fbcc1450c913e120b6bae542f" member_type: VOTER last_known_addr { host: "127.5.241.124" port: 33113 } } }
I20260619 02:44:06.336771 28920 sys_catalog.cc:458] T 00000000000000000000000000000000 P c37b3a825dd24050ab0b625340a2ffdd [sys.catalog]: This master's current role is: LEADER
I20260619 02:44:06.337423 28927 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:44:06.338143 28927 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:44:06.357493 28927 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c37b3a825dd24050ab0b625340a2ffdd: Generated new TSK 0
I20260619 02:44:06.357767 28927 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 02:44:06.382593 28741 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:44:06.380944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34828 (local address 127.5.241.126:45905)
0619 02:44:06.380999 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:06.381001 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:06.381924 (+ 923us) server_negotiation.cc:407] Connection header received
0619 02:44:06.382040 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:06.382042 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:06.382053 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:06.382083 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:06.382527 (+ 444us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34828: BlockingRecv error: recv got EOF from 127.0.0.1:34828 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260619 02:44:06.383494 28816 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:44:06.381334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53134 (local address 127.5.241.125:39683)
0619 02:44:06.381373 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:06.381375 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:06.382990 (+ 1615us) server_negotiation.cc:407] Connection header received
0619 02:44:06.383085 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:06.383086 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:06.383097 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:06.383125 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:06.383430 (+ 305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53134: BlockingRecv error: recv got EOF from 127.0.0.1:53134 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260619 02:44:06.384351 28892 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:44:06.381659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37814 (local address 127.5.241.124:33113)
0619 02:44:06.381707 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:06.381709 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:06.383806 (+ 2097us) server_negotiation.cc:407] Connection header received
0619 02:44:06.383914 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:06.383915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:06.383927 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:06.383967 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:06.384289 (+ 322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37814: BlockingRecv error: recv got EOF from 127.0.0.1:37814 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260619 02:44:06.386826 28816 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:44:06.385999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53144 (local address 127.5.241.125:39683)
0619 02:44:06.386049 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:06.386051 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:06.386266 (+ 215us) server_negotiation.cc:407] Connection header received
0619 02:44:06.386376 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:06.386377 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:06.386394 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:06.386424 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:06.386749 (+ 325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53144: BlockingRecv error: recv got EOF from 127.0.0.1:53144 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260619 02:44:06.389986 28816 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:44:06.389135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53152 (local address 127.5.241.125:39683)
0619 02:44:06.389166 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:06.389168 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:06.389490 (+ 322us) server_negotiation.cc:407] Connection header received
0619 02:44:06.389587 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:06.389588 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:06.389598 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:06.389626 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:06.389930 (+ 304us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53152: BlockingRecv error: recv got EOF from 127.0.0.1:53152 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260619 02:44:06.393862 28816 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:44:06.393161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53162 (local address 127.5.241.125:39683)
0619 02:44:06.393191 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:06.393193 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:06.393395 (+ 202us) server_negotiation.cc:407] Connection header received
0619 02:44:06.393497 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:06.393498 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:06.393506 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:06.393533 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:06.393824 (+ 291us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53162: BlockingRecv error: recv got EOF from 127.0.0.1:53162 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260619 02:44:06.399807 28816 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:44:06.399072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53164 (local address 127.5.241.125:39683)
0619 02:44:06.399105 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:06.399106 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:06.399336 (+ 230us) server_negotiation.cc:407] Connection header received
0619 02:44:06.399432 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:06.399433 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:06.399442 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:06.399471 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:06.399766 (+ 295us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53164: BlockingRecv error: recv got EOF from 127.0.0.1:53164 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260619 02:44:06.409873 28816 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:44:06.409049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53168 (local address 127.5.241.125:39683)
0619 02:44:06.409087 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:06.409089 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:06.409332 (+ 243us) server_negotiation.cc:407] Connection header received
0619 02:44:06.409431 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:06.409432 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:06.409442 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:06.409474 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:06.409809 (+ 335us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53168: BlockingRecv error: recv got EOF from 127.0.0.1:53168 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260619 02:44:06.428404 28816 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:44:06.427440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53182 (local address 127.5.241.125:39683)
0619 02:44:06.427484 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:06.427486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:06.427770 (+ 284us) server_negotiation.cc:407] Connection header received
0619 02:44:06.427873 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:06.427874 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:06.427887 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:06.427916 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:06.428342 (+ 426us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53182: BlockingRecv error: recv got EOF from 127.0.0.1:53182 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260619 02:44:06.462481 28816 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:44:06.461740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53194 (local address 127.5.241.125:39683)
0619 02:44:06.461787 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:06.461789 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:06.461919 (+ 130us) server_negotiation.cc:407] Connection header received
0619 02:44:06.461991 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:06.461992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:06.462004 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:06.462075 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:06.462407 (+ 332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53194: BlockingRecv error: recv got EOF from 127.0.0.1:53194 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260619 02:44:06.528198 28816 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:44:06.527562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53196 (local address 127.5.241.125:39683)
0619 02:44:06.527628 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:06.527630 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:06.527664 (+ 34us) server_negotiation.cc:407] Connection header received
0619 02:44:06.527739 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:06.527740 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:06.527750 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:06.527816 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:06.528135 (+ 319us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53196: BlockingRecv error: recv got EOF from 127.0.0.1:53196 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260619 02:44:06.658126 28816 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:44:06.657491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53208 (local address 127.5.241.125:39683)
0619 02:44:06.657561 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:06.657563 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:06.657574 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:44:06.657614 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:06.657616 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:06.657627 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:06.657702 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:06.658074 (+ 372us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53208: BlockingRecv error: recv got EOF from 127.0.0.1:53208 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260619 02:44:06.916042 28816 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:44:06.915435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53210 (local address 127.5.241.125:39683)
0619 02:44:06.915514 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:06.915516 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:06.915524 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:44:06.915552 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:06.915553 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:06.915562 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:06.915628 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:06.915970 (+ 342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53210: BlockingRecv error: recv got EOF from 127.0.0.1:53210 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 02:44:07.300112 28919 catalog_manager.cc:1259] Loaded cluster ID: bfc14547cf3f4eb1b8b5012d8ea61dab
I20260619 02:44:07.300182 28919 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 34f5d73fbcc1450c913e120b6bae542f: loading cluster ID for follower catalog manager: success
I20260619 02:44:07.300980 28919 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 34f5d73fbcc1450c913e120b6bae542f: acquiring CA information for follower catalog manager: success
I20260619 02:44:07.301290 28919 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 34f5d73fbcc1450c913e120b6bae542f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 02:44:07.332202 28953 catalog_manager.cc:1259] Loaded cluster ID: bfc14547cf3f4eb1b8b5012d8ea61dab
I20260619 02:44:07.332270 28953 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 282572e78c2a41bcbd79dd37da9df53b: loading cluster ID for follower catalog manager: success
I20260619 02:44:07.333073 28953 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 282572e78c2a41bcbd79dd37da9df53b: acquiring CA information for follower catalog manager: success
I20260619 02:44:07.333370 28953 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 282572e78c2a41bcbd79dd37da9df53b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 02:44:07.431526 29020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:07.429150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53222 (local address 127.5.241.125:39683)
0619 02:44:07.429258 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:07.429260 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:07.429274 (+ 14us) server_negotiation.cc:407] Connection header received
0619 02:44:07.429341 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:07.429342 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:07.429392 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:07.429466 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:44:07.429583 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:07.430073 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:07.430822 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:07.431013 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:07.431083 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:07.431201 (+ 118us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:44:07.431310 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:44:07.431368 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:07.431396 (+ 28us) server_negotiation.cc:299] Negotiation successful
0619 02:44:07.431413 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":55,"threads_started":1}
I20260619 02:44:07.435539 29026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:07.433299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37818 (local address 127.5.241.124:33113)
0619 02:44:07.433383 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:07.433385 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:07.433449 (+ 64us) server_negotiation.cc:407] Connection header received
0619 02:44:07.433504 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:07.433505 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:07.433538 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:07.433601 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:44:07.433685 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:07.434160 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:07.434860 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:07.435046 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:07.435093 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:07.435248 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:44:07.435344 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:44:07.435399 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:07.435431 (+ 32us) server_negotiation.cc:299] Negotiation successful
0619 02:44:07.435446 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65,"thread_start_us":43,"threads_started":1}
I20260619 02:44:07.439960 29026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:07.437395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37822 (local address 127.5.241.124:33113)
0619 02:44:07.437434 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:07.437436 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:07.437443 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:44:07.437468 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:07.437469 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:07.437480 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:07.437682 (+ 202us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:44:07.437715 (+ 33us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:07.438226 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:07.439175 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:07.439468 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:07.439524 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:07.439691 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:44:07.439726 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:44:07.439781 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:07.439843 (+ 62us) server_negotiation.cc:299] Negotiation successful
0619 02:44:07.439881 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:44:07.440351 29020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:07.437202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53234 (local address 127.5.241.125:39683)
0619 02:44:07.437323 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:07.437325 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:07.437514 (+ 189us) server_negotiation.cc:407] Connection header received
0619 02:44:07.437598 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:07.437599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:07.437610 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:07.437680 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:44:07.437781 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:07.438492 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:07.439443 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:07.439720 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:07.439789 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:07.440093 (+ 304us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:44:07.440129 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:44:07.440165 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:07.440252 (+ 87us) server_negotiation.cc:299] Negotiation successful
0619 02:44:07.440275 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260619 02:44:07.440619 29033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:07.437215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34842 (local address 127.5.241.126:45905)
0619 02:44:07.437373 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:07.437375 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:07.437456 (+ 81us) server_negotiation.cc:407] Connection header received
0619 02:44:07.437538 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:07.437539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:07.437618 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:07.437696 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:44:07.437856 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:07.438593 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:07.439498 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:07.439822 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:07.439883 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:07.440094 (+ 211us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:44:07.440201 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:44:07.440261 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:07.440494 (+ 233us) server_negotiation.cc:299] Negotiation successful
0619 02:44:07.440537 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":62,"threads_started":1}
I20260619 02:44:07.442194 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 28669
I20260619 02:44:07.447345 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 28744
I20260619 02:44:07.452451 6085 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskflgrG6/build/release/bin/kudu with pid 28819
2026-06-19T02:44:07Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2802 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5612 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (274606 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST